Article Text

Download PDFPDF

Alarm system management: evidence-based guidance encouraging direct measurement of informativeness to improve alarm response
  1. Michael F Rayo1,
  2. Susan D Moffatt-Bruce2
  1. 1Department of Quality and Patient Safety, The Ohio State University, Columbus, Ohio, USA
  2. 2Department of Thoracic Surgery, College of Medicine, The Ohio State University, Columbus, Ohio, USA
  1. Correspondence to Dr Michael F Rayo, The Ohio State University Wexner Medical Center, Doan Hall 0130, 410 W 10th Ave, Columbus, OH 43210, USA; mike.rayo{at}osumc.edu

Abstract

Although there are powerful incentives for creating alarm management programmes to reduce ‘alarm fatigue’, they do not provide guidance on how to reduce the likelihood that clinicians will disregard critical alarms. The literature cites numerous phenomena that contribute to alarm fatigue, although many of these, including total rate of alarms, are not supported in the literature as factors that directly impact alarm response. The contributor that is most frequently associated with alarm response is informativeness, which is defined as the proportion of total alarms that successfully conveys a specific event, and the extent to which it is a hazard. Informativeness is low across all healthcare applications, consistently ranging from 1% to 20%. Because of its likelihood and strong evidential support, informativeness should be evaluated before other contributors are considered. Methods for measuring informativeness and alarm response are discussed. Design directions for potential interventions, as well as design alternatives to traditional alarms, are also discussed. With the increased attention and investment in alarm system management that alarm interventions are currently receiving, initiatives that focus on informativeness and the other evidence-based measures identified will allow us to more effectively, efficiently and reliably redirect clinician attention, ultimately improving alarm response.

  • Evidence-based medicine
  • Human factors
  • Interruptions

Statistics from Altmetric.com

Introduction

Although there is strong consensus on the importance of implementing interventions to reduce ‘alarm fatigue’, study findings are often unclear if a given intervention impacts the central purpose of an alarm system, to ‘redirect our attention from something that is less important to something that is more important.’1 A confluence of factors have created powerful incentives for US hospitals to create clinical alarm management programmes, including The Joint Commission's (TJC) 2014 National Patient Safety Goal (NPSG) 6,2 increased visibility of the hazards of clinical alarms,3 and the increasing number of alarm-related deaths reported to the Food and Drug Administration (FDA)4 and TJC.5 However, the alarm fatigue label subsumes a myriad of potential contributors, and only some have been shown to adversely affect alarm system performance in either laboratory or real-world settings. There is little guidance on how to determine to what extent each of these phenomena is contributing to the observed problems and, therefore, what the focus of alarm system interventions should be. After summarising the relevant literature regarding alarm fatigue, we will discuss the most commonly reported contributors thought to impact the overall system's response to hazardous events, share common techniques for collecting data for the most likely contributor, informativeness, and discuss potential strategies for designing new interventions. In this way, we aim to produce guidance on how to effectively diagnose and treat a specific institution's alarm problem, and which measures to use in order to assess whether or not the health of the overall system has improved.

Reported causes of alarm fatigue, and relationship to alarm response

Alarm fatigue is an umbrella term meant to encompass all phenomena that are responsible for clinicians’ increased response time and decreased response rate to alarms. It is reported as the cause when clinicians improperly ignore, override, silence, or mute clinical alarms that signify critical patient events.6 This decreased response rate to alarms has been associated with patients’ deaths,1 ,7 as well as permanent loss of function, and prolonged hospitalisation.8 However, of the many phenomena that are thought of as part of alarm fatigue, only some have been shown to predict the alarm system's ability to redirect attention to emerging hazards and, therefore, improve patient outcomes.

Contributors unsupported by laboratory or real-world studies

Much of the healthcare literature includes clinician motivation, self-discipline and commitment as potential causes of alarm fatigue.8 ,9 This has led some to the conclusion that, although it is not technically feasible now, these systems would be safer without human beings in them.10 Others posit that training to counteract these psychological causes would result in a better alarm system safety.9 A similar belief is that the clinician alarm response is due, in part, to a type of apathy or fatigue that sets in due to an overabundance of alarms that overwhelms and desensitises clinicians.6 ,9 ,11–13

However, although each hypothesis has high face validity, no descriptive or comparative studies have been conducted to determine if these phenomena directly impact overall health of the system. Regarding the clinicians being inherently unsafe, there is extensive evidence across multiple industries to the contrary showing that humans create safety in complex sociotechnical systems by filling in the inherent safety gaps in underspecified procedures and brittle technologies.14 Regarding training and increasing staff commitment, no studies have shown that these types of interventions have resulted in sustained improvements in alarm response. Regarding the overabundance of alarms, although many studies report the rate of total alarms and reductions resulting from interventions, there is nothing in the literature that gives generic or specific guidance on acceptable or dangerous rates. Additionally, there is no indication that these alarms, by themselves, are outside of theoretical safety envelopes with respect to sound intensity15 or visual salience.16 Although it is possible that the reduction of alarm rate equates to a reduction in false alarms or a meaningful reduction in mental workload, which have both been shown to improve alarm response, these relationships are not explicitly examined. Additionally, reducing the total rate of alarms without explicitly measuring the false alarm rate, risks reducing true positive alarms as well.

Contributors supported by laboratory or real-world studies

In addition to the reported contributors mentioned above, there are many contributors to alarm response that are supported in the literature. The evidence shows that all alarm systems are prone to a variety of well-understood and well-studied technical problems. Many of these issues affect the clinicians working in these environments, as well as the patients and their families.6

The most prevalent of these is low informativeness of a set of alarms. Informativeness is the discrimination power of an alarm system to detect abnormalities in the world and infer what is worthy of attention.17 ,18 It measures the proportion of alarm signals that successfully convey a specific hazard,17 which requires a combination of the sensory, informational, attentional and cognitive aspects described in table 1. Informativeness drops sharply when alarm systems increasingly notify clinicians of events that are not occurring (ie, false alarms18), and of events that are occurring but are not hazards (ie, unnecessary alarms, also called non-actionable or nuisance alarms6 ,18). Group alarms also cause informativeness to drop, when one signal refers to more than one hazard or more than one hazard severity.19 It has been shown mathematically,20 and in multiple studies, that decreased informativeness is associated with a decreased response rate and increased response time for all operators, human or otherwise.21 ,22 This effect is attenuated if the operator understands why the alarm is sounding.4 ,18 This research reframes alarm fatigue as a necessary calibration by the human operator to pay decreasing amounts of attention to signals that do not merit that attention.19

Table 1

Important aspects of alarm systems based on human factors, engineering, and experimental psychology literature

Increased mental workload has also been associated with a deterioration in alarm response.7 ,21 Increasing mental workload decreases the likelihood of attending to emerging, external stimuli. In high-tempo situations with increased mental workload, clinicians in high-tempo situations regularly ignored alarms that were regularly attended to in low-tempo situations.23 ,24 Pilots, during periods of high workload, reduced their field of vision to preserve performance on tasks directly related to flying.25 Drivers, as driving difficulty increased, also reduced their field of view, ignoring peripheral signs, alarms and warnings.26 ,27

It has also been shown that the acoustical design of alarms can produce signals that are not detected, or to induce negative physiological responses. This can occur due to signals that are overly intense,15 ,28 physiologically unpleasant due to frequency,29 startling due to sudden onset,28 masked by other signals in the intended environment,19 ,28 ,29 or are insufficient to determine the location of the source30 or intended urgency.31 Finally, these alarms can create memory burdens, making it difficult to learn and remember which alarms signify which events.29 ,32 ,33

Ultimately, these technical problems make it increasingly difficult for clinicians to effectively detect abnormalities requiring their attention and reprioritise their actions. The evidence shows that it is these technical problems, and not motivational or volitional issues, that are the most likely contributors to alarm-related issues.19 ,20 If they are not examined and ruled out before issues of motivation or volition are posited, researchers and clinicians risk solving the wrong problem.

Where to start? Informativeness

Although there are many potential contributors to a specific alarm response problem, it is likely that informativeness plays a prominent role. Across all healthcare settings, informativeness consistently ranges from 1% to 20%.6 Across multiple studies of electronic health records, alerts’ informativeness ranged from 1% to 12%.34–36 In separate studies of haemodynamical and ventilator and pulse oximetry alarms, informativeness was 20%37 and 23%,38 respectively. In a national survey performed in 2005–2006, 78% respondents stated that alarms were often overridden due to distrust.39 Using findings from Bliss,22 these rates predict the high proportion of alarms that are overridden or ignored.23 ,36 ,40 These levels of informativeness are well beneath the tentative theoretical threshold of 71% that Wickens and Dixon38 suggested that alarms need in order to be more helpful than harmful to the overall system.

Collecting data to measure informativeness requires more data, and is, therefore, more labour intensive than some of the other measures. Besides knowing the event that the alarm is signifying, context of the alarm and of the patient immediately preceding and following the alarm must also be known. The waveform representation of the sensor data that is used to generate the alarm can be used to determine the presence of artefact and, therefore, false alarms.41 Additionally, video footage or direct observation of patients can be used to determine if the event signified by the alarm is occurring.42 Clinical experts are used to determining whether the alarm was clinically relevant (ie, not unnecessary) based on the patient's recent activity and history.42–45 Informativeness is most often measured as positive predictive value (PPV), which measures the proportion of true positives out of all positive responses,6 ,17 but have also been measured by negative predictive value, sensitivity and specificity,45 and could be measured by any measure of discrimination power. Most studies calculate PPV based only on the traditional definition of false alarms, but there is recent awareness that requiring that true positives be neither false nor unnecessary may be a better predictor of alarm response in some settings.18 ,46

Although improving informativeness will likely improve alarm response, it is possible that other contributors in the setting are the more substantial contributors. It is prudent to consider the likelihood of the other evidence-based factors as well, many of which are noted above. Being aware of these other factors and the current evidence-based guidance for each, where applicable, may reveal potential contributors that otherwise would have been obscured. Finally, it is also prudent to assess response time and response rate directly, to ensure that the current alarm response requires intervention, and that the intervention results in response times and rates within acceptable safety boundaries. Alarm response can sometimes be measured without direct observation if it can be discerned from the technology, for example, when it requires clinicians to explicitly override guidance and give a reason.36 Most often, though, the machine data that is available will not contain sufficient context to make a determination. In these cases, direct observation of the clinician or unit is necessary, recording whether each alarm was attended to or not, and if corrective action was taken. A good example is given by Seagull and Sanderson.23

In our institution, the alarm management team uses a triangulation of three measures to determine the relative priority of multiple competing needs for intervention. Response times to the most important alarm signals, as were determined per TJC NPSG 6,2 are collected electronically via a Secondary Alarm Notification System, which also serves as an alarm aggregator. From these data, units are selected for direct observation, both to measure informativeness and alarm response with the necessary context. Analysis of these data and consideration of potential other contributors all drive the ultimate design of interventions. Based on this analysis, the intervention may be to change default alarm thresholds, allow more clinician discretion in setting thresholds for a given patient, procure sensors with better discrimination power, or procure software that can use data from multiple parameters, trends over time, and signal quality to reduce false alarms. Direct observation is then conducted to determine if informativeness and alarm response have improved.

Alternatives when traditional alarms are insufficient

In addition to improving these evidence-based measures on existing alarm systems, the literature offers guidance on alternatives for information representation and sensory modalities. Often, these are most valuable when the discrimination power of available sensors and software, or the auditory characteristics of the environment, make traditional audible alarms ineffective or harmful to overall system performance.47 In these situations, alternative visual, auditory, tactile and olfactory displays are recommended. Using alternative sensory modalities can make use of less saturated sensory channels, thereby reducing overall mental workload,48 and can leverage the unique benefits of each modality. Using alternative representations, especially those containing continuous non-interrupting data, can increase informativeness and reduce the overall mental workload required for directing attention and interpretation.49–52 The continuous beep of an echocardiogram that signifies heart rate is a good example of this. This always-on signal allows the listener to detect change, not by commanding attention on onset, but by presenting a continuous baseline that new events are subconsciously compared with. Additionally, visual analogue representations can reveal trends and other data relationships that traditional threshold alarms often obscure.49 ,51 Although these alternate modalities and representations are not optimal in all situations,49 their benefits should be considered as part of an optimal alarm system design process.

With the increased attention and investment that clinical alarm management is currently receiving, it is more critical now than ever to adopt a pragmatic and rigorous approach to diagnose and treat the alarm-related issues that we face in our organisations. That approach should systematically examine and rule out all the known contributors that are likely responsible for the observed symptoms before considering those that are less likely. It is our intent to increase awareness and use of these generalised models, although we understand that they may need to be refined further for specific healthcare settings. In this way, we can more effectively, efficiently and reliably redirect clinician attention, ultimately improving alarm response.

Acknowledgments

The authors wish to thank David D Woods, PhD (Department of Industrial and Systems Engineering, The Ohio State University), for providing conceptual support and editorial review.

References

Footnotes

  • Twitter Follow Michael Rayo at @hepcatrayo

  • Contributors MFR manuscript writing. SDM-B critical revision of the manuscript.

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.