ICUCollection of annotated data in a clinical validation study for alarm algorithms in intensive care—a methodologic framework☆
Introduction
Monitoring of physiologic functions in critical ill patients by alarm systems is crucial in intensive care units (ICUs). However, there are several factors reducing the clinical value of the currently used alarm systems. In a well-appointed ICU, more than 40 different audible alarms may occur [1] and only 50% of the critical alarms are correctly identified by the staff [2]. The noise peaks frequently exceed 80 dB [3] and are comparable to the noise levels next to busy streets, thereby generating stress for patients and potential health hazards to employees [4], [5]. In addition, there is a substantial risk that the ICU staff may match their response rates to the probability of alarms with a clinical consequence, which would likely result in a missing response to relevant alarms [6].
As a consequence, one goal is the development of monitoring systems with fewer alarms but with preserved sensitivity for clinically relevant alarms. Most current alarm systems are able to produce 3 types of alarms as follows: threshold alarms, arrhythmia alarms, and technical alarms. Threshold alarms are based on a comparison of each data value with fixed alarm limits and occur most frequently. Several studies showed that up to 90% of these alarms are without clinical relevance and mainly due to artifacts [7], [8], [9], [10], implying that the development and implementation of new methods that modify or replace threshold alarms have the potential to reduce the number of alarms.
Various approaches from the areas of statistics and artificial intelligence have been proposed for the improvement of existing alarm systems (overview in Imhoff and Kuhls [11]). We currently evaluate a new alarm procedure with the aim of reducing the number of false alarms.
Prerequisite for the validation of these new alarm algorithms is the extraction of continuous physiologic data, monitor alarms, and alarm settings from the surveillance network. Afterward, the monitor alarms are annotated with respect to the clinical relevance and technical validity by an experienced physician to create a reference database. The new alarm algorithms will be validated off-line on the basis of this annotated database, allowing us to calculate sensitivity and specificity as measures of performance for the comparison of the 2 alarm procedures. The reference database will also enable a thorough analysis of the currently used monitoring system with respect to the frequency and reliability of alarms and causes for false alarms.
This article focuses on the data collection and clinical annotation concept and compares it with other studies in this area. In addition, preliminary results from the ongoing study are presented.
Section snippets
Data collection
This study was approved by the Ethics Committee of the University of Regensburg (Regensburg, Germany) and performed in accordance with the Declaration of Helsinki. Data collection is taking place at the 12-bed medical ICU.
Patients with monitoring of at least heart rate, invasive or closely monitored noninvasive arterial blood pressure, and oxygen saturation are included in the study. Patients for whom a frequent occurrence of alarms can be expected are preferred. The following data are
Patient characteristics
At this stage of the study, the reference database contains 515 annotated hours of monitoring for 38 different patient cases. Mean age of all included patients was 63 ± 13 years (minimum, 36 years; maximum, 90 years). Of the patients, 71.1% underwent artificial ventilation and 63.2% were in need of catecholamine therapy. As a criterion for the patient's severity of illness, we calculated the SAPS II (simplified acute physiology score) score at time of admission to the ICU (mean value, 53.4 ±
Discussion
This article presents a system for collecting real-time bedside monitoring data in conjunction with annotations of clinically relevant events by means of video monitoring. The collected data establish a basis for the development and evaluation of “smart” alarm algorithms.
Several studies involving the collection of ICU monitoring data have been published. In most cases, these data were obtained by connecting a computer at the bedside to the patient monitoring via a serial line [12], [13]. In our
Conclusion
We have presented a new system for collecting ICU monitoring data in conjunction with clinical annotations to form a database. This database will be used for the evaluation of the current alarm system and provides the basis for testing of new alarm algorithms. The preliminary results indicate that only a small portion of alarms are indeed alarm-relevant, emphasizing the clinical importance of a smart monitoring system with a reduced number of false alarms.
References (13)
- et al.
Name that tone. The proliferation of alarms in the intensive care unit
Chest
(1994) - et al.
Identification and modification of environmental noise in an ICU setting
Chest
(1998) Alarms in the intensive care unit: how can the number of false alarms be reduced?
Crit Care
(2001)- et al.
Noise in the ICU
Intensive Care Med
(1993) - et al.
Noise in critical care
Care Crit Ill
(2007) - et al.
Human probability matching behaviour in response to alarms of varying reliability
Ergonomics
(1995)
Cited by (63)
Nurses’ responses to monitor alarms in an intensive care unit: An observational study
2020, Intensive and Critical Care NursingCitation Excerpt :Nurses’ responses were grouped under seven categories: ‘patient assessment and observation’, ‘responding to the patient’s complaint’, ‘informing the doctor of the patient’s condition’, ‘positioning the patient’, ‘checking and resolving contact and transmission problems’, ‘checking and adjusting device settings’ and ‘silencing the alarm’. An alarm classification and identification list was generated according to literature to collect data more systematically (Bitan et al., 2004; Bridi et al., 2014; Gazarian, 2014; Görges et al., 2009; Rayo et al., 2016; Siebig et al., 2010). The alarms were divided into three categories:
Suppression of noises using fast independent component analysis (FICA) and signal saturation using fuzzy adaptive histogram equalization (FAHE) for intensive care unit false alarms
2019, Measurement: Journal of the International Measurement ConfederationAdvanced Technology in Pediatric Intensive Care Units. Have They Improved Outcomes?
2016, Pediatric Clinics of North AmericaMeta-analysis on the Prevalence and Impact of False, Nonactionable, and Nuisance Alarms in Critical Care Environments: Informing Evidence-Based Practices for Improving Patient Safety in Alarm Monitoring Systems
2023, BMEiCON 2023 - 15th Biomedical Engineering International ConferenceSemi-Supervised Learning for Sparsely-Labeled Sequential Data: Application to Healthcare Video Processing
2023, Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023
- ☆
The work presented has been supported by the Deutsche Forschungsgemeinschaft (SFB 475, Reduction of complexity in multivariate data structures) in cooperation with the TAHI-section of the ESICM.
- 1
Authors contributed equally to this manuscript.