ResearchEvaluating Implementation of the Emergency Severity Index in a Belgian Hospital
Section snippets
Design, Setting, and Participants
This cross-sectional observational study was conducted at the emergency department of a 1900-bed tertiary care teaching hospital in Belgium. The emergency department has an annual census of approximately 54 000 patients and a census of 102 to 210 patients per day. Between 2005 and 2008, the patient volume increased by approximately 3814 patients (8%). The ESI was implemented in the emergency department in January 2009. The ED staff had no previous experience with the ESI or triage in general.
Results
The mean score for the survey was 23/30. Overall exact agreement between triage nurses and reference answers was 77.5%. The mistakes made by triage nurses resulted more commonly in undertriage (77.5%) than overtriage. Undertriage ranged from a difference of 1, 2, or 3 levels from the true ESI level; overtriage ranged from a difference of 1 or 2 levels from the true ESI level. The unweighted κ score was 0.72 (95% confidence interval [CI] 0.69-0.74), κlw was 0.84 (95% CI 0.77-0.91), κqw was 0.92
Discussion
Triage is a vital tool in preventing unsafe waiting times for the most vulnerable patients. However, to accomplish this goal, the implemented triage system needs to be valid and reliable and staff need to use the tool correctly. Because triage relies on a nurse’s interpretation of the level specific criteria, interrater variability is problematic but surmountable. In other words, the interrater agreement of a triage system is dependent upon nurses’ knowledge and experience, as well as their
Limitations
When interpreting the results of this study, one needs to be aware of several limitations. First, this study was conducted at a single center with a limited number (N = 52) of triage nurses. Second, because the native language of triage nurses is Dutch, the implementation team had to translate the contents of the implementation manual, and thus potential important nuances could have been lost. Third, triage decision making is a process influenced by multiple factors.12, 13, 14, 15, 16, 17, 18,
Implications for Emergency Nurses
The results of this study suggest that Belgian nurses have difficulties with the interpretation of high acuity level criteria, as well as estimating the number of resources a patient needs to reach a dispositional decision. The ESI manual noted that these decisions are primarily based on nurses’ knowledge and experience. If a triage system is largely based on knowledge and experience, interobserver variation is unavoidable, which implies that sufficient attention must be directed at the
Conclusions
In conclusion, implementation of the ESI into a novel setting guided by a locally developed training program resulted in suboptimal interrater agreement of triage classification. The majority (77.5%) of the assigned triage codes by nurses were in concordance with the true ESI level, and errors were made in all ESI levels, with the highest number of errors in ESI level 2. Further, this study shows that existing weighted κ schemes overestimate the interrater agreement between of the triage
Jochen Bergs is PhD candidate, Patient Safety Research Group, Faculty of Business Economics, Hasselt University, Hasselt, Belgium.
References (27)
- et al.
Reevaluating triage in the new millennium: a comprehensive look at the need for standardization and quality
J Emerg Nurs
(1999) - et al.
The influence of triage systems and triage scores on timeliness of ED analgesic administration
Am J Emerg Med
(2008) - et al.
Triage of geriatric patients in the emergency department: validity and survival with the Emergency Severity Index
Ann Emerg Med
(2007) - et al.
The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption
J Emerg Nurs
(2004) - et al.
Adjusting weighted kappa for severity of mistriage decreases reported reliability of emergency department triage systems: a comparative study
J Clin Epidemiol
(2009) - et al.
Calculating kappas from adjusted data improved the comparability of the reliability of triage systems: a comparative study
J Clin Epidemiol
(2010) - et al.
Myths versus facts in emergency department overcrowding and hospital access block
Med J Aust
(2009) - et al.
Access block and overcrowding: a literature review
(2009) - et al.
Emergency department overcrowding following systematic hospital restructuring: trends at twenty hospitals over ten years
Acad Emerg Med
(2001) Increase in patient mortality at 10 days associated with emergency department crowding
Med J Aust
(2006)
The association between hospital overcrowding and mortality among patients admitted via Western Australian emergency departments
Med J Aust
Hospital overcrowding: a threat to patient safety?
Med J Aust
Urgency classification methods for emergency department visits: do they measure up?
Pediatr Emerg Care
Cited by (18)
Evaluating the effects of triage education on triage accuracy within the emergency department: An integrative review
2023, International Emergency NursingEducational intervention in triage with the Swedish triage scale RETTS©, with focus on specialist nurse students in ambulance and emergency care – A cross-sectional study
2022, International Emergency NursingCitation Excerpt :The basic educational intervention in this study was brief (4 h) and included recorded lectures and triaging five paper-based scenarios in groups under teacher-led review with focus on basic application rather than deeper reflection on the triage assessment when using RETTS©. In comparison, the implementation of ESI through education consists of a 3-hour theoretical presentation followed by practice sessions on paper-based scenarios, but this path too results in a suboptimal inter-rater agreement of triage classification [48]. Only 4 out of 46 scenarios in this study showed complete agreement, and most scenarios showed some dispersion often across the border between stable and unstable.
Reliability and validity of the four-level Chinese emergency triage scale in mainland China: A multicenter assessment
2020, International Journal of Nursing StudiesCitation Excerpt :Estimates of medical resources available always vary between triage nurses and physicians from different EDs in China. Further, there are reports of the ESI's suboptimal performance, with low accuracy and high variability, outside the United States (Bergs et al., 2014; Jordi et al., 2015). Another factor is the complexities of triage systems in terms of the different presenting complaints (Karjala and Eriksson, 2017).
Comparison between Emergency Severity Index plus peak flow meter and Emergency Severity Index in the dyspneic patients with chronic obstructive pulmonary disease: A randomized clinical trial
2019, Turkish Journal of Emergency MedicineCitation Excerpt :High-risk criteria cited on ESI level 2 are dependent on nurses’ knowledge of COPD, so this may become a source of discrepancy in triage decisions. In this regard, Bergs et al. showed that the majority of the mistriages that occurred on ESI level 2 were undertriage.17 High-risk criteria in the ESI may be misinterpreted by triage nurses,18 who differ in terms of work experience, knowledge, and skills.
Comparison of the Emergency Severity Index versus the Patient Acuity Category Scale in an emergency setting
2018, International Emergency NursingCitation Excerpt :The low number of patients in P4 may indicate that this category may no longer be adequate for use in Singapore. For patient case scenarios, both the ESI and the PACS showed very good quadratic weighted κ values, which were comparable with those in the literature [11,29]. For actual patients, the quadratic weighted κ for the ESI was moderate and similar to previous studies [10,12].
Nursing Perceptions of the Emergency Severity Index as a Triage Tool in the United Arab Emirates: A Qualitative Analysis
2018, Journal of Emergency Nursing
Jochen Bergs is PhD candidate, Patient Safety Research Group, Faculty of Business Economics, Hasselt University, Hasselt, Belgium.
Sandra Verelst is Emergency Physician, Department of Emergency Medicine, Leuven University Hospitals, Leuven, Belgium.
Jean-Bernard Gillet is Professor, Department of Emergency Medicine, Leuven University Hospitals, Leuven, Belgium.
Dominique Vandijck is Professor, Patient Safety Research Group, Faculty of Business Economics, Hasselt University, Hasselt, Belgium, and Professor, Department of Public Health and Health Economics, Faculty of Medicine and Health Sciences, Ghent University, Ghent, Belgium.