Towards a framework to select techniques for error prediction: Supporting novice users in the healthcare sector
Introduction
Incident investigation techniques have been increasingly used in healthcare and are supported by literature specifically written to help novices in choosing techniques (Johnson, 2003). However, a similar pattern has not occurred for predictive safety techniques. Despite many decades of acceptance of the predictive safety techniques in other industries, Lyons et al. (2004a) found only seven techniques had been published as being used for healthcare application (change analysis, FMEA, HAZOP, influence diagrams, SHERPA, event trees and fault trees). This was particularly noteworthy when task analysis – the precursory step for many human reliability analysis techniques – has been applied to several areas within healthcare.
The reasons for its narrow application may be the lack of awareness that there are so many usable techniques or due to the challenge of choosing between the overwhelming number of techniques – with 520 safety assessment methodologies identified for supporting air traffic management (Everdij, 2004). Although awareness and understanding of the practical application of such a great number of techniques may appear impossible, it should be emphasised that not all the techniques are discrete, with many variants evolving for a subset of the techniques. One idiosyncrasy of safety assessment techniques is that there is a popular trend to give the techniques acronymic (HEART) or initialistic (HTA) names. Therefore, often techniques that are identical in form have been given different names due to application in different domains or have minor changes made by authors. Conversely identical techniques have evolved with slightly different names – e.g. Safety barrier function analysis (Kecklund et al., 1996), accident evolution barrier analysis (Svenson, 1991, Svenson, 2001), energy barrier analysis (Rahimi, 1986) and barrier analysis (Hollnagel, 2004). Kirwan (1998a) outlines an approximate evolution relationship of many of these techniques (e.g. FMEA, HAZOP and event tree based techniques) including cross-links between these.
Even in industries more familiar with reliability engineering techniques, it has been speculated that these techniques were used so scarcely due to the unavailability of the means and/or techniques, technique complexity (Paz Barroso and Wilson, 2000) or lack of information on resources required (Ainsworth and Marshall, 1998). Pradhan et al. (2001) suggest the cultural norms of healthcare also add to the challenges – perhaps resulting in demands for tailor-made instead of the industry standard techniques.
The continuous professional development required by healthcare professionals is broader than the remits of safety assessment. Even with governmental support, this may limit opportunities to learn a range of safety techniques. It may be preferable for healthcare users unfamiliar with the field to choose a technique to solve a specific problem and then target the technique education accordingly.
Otherwise, there is a danger that novices are using techniques to guide safety-related decisions without the training appropriate to gain the validity and reliability assumed from the technique (Stanton and Young, 2003).
Whilst there have been a number of reviews of the human reliability analysis and similar techniques (Everdij, 2004; Humphreys, 1988; Suokas, 1988, Kirwan, 1992a, Kirwan, 1992b, Kirwan, 1994, 1998; Stanton et al., 2005, Wreathall and Nemeth, 2004), most have focussed more on the accuracy and reliability of the techniques rather than the requirements of the problem to be analysed. Together with the issues of a lack of technique awareness and expertise, the pressures on healthcare personnel will tend to make healthcare professionals more reluctant to invest in the rigorous education requirements for the more complex techniques, designed to produce more accurate results. This may lead to personnel choosing the “quick and dirty” techniques even if they are not the most appropriate for the problem. Furthermore, the previous reviews have always been written in a technique-by-technique format thus demanding time commitment in investigating techniques that would not be chosen, simply to identify the technique that best matches the features of the chosen problem. In many cases, this assumes the readers to have some knowledge or experience in the field and already have a mental model of many or all the techniques. In short, what is required is a user-focussed approach for selecting a technique – namely a bottom-up method of selection based on the resources, constraints and requirements of the user.
As yet, there has been no dedicated study to research the usability of the techniques, the limits of requirements for each technique in terms of time or human resources or any efforts to support the novice in selecting a technique to analyse their problem. Therefore, this paper provides a first step in providing a literature-based framework for selection of techniques.
Section snippets
Terms and definitions
Sources of confusion are the terms and definitions used when describing the techniques. Therefore this section provides a cautionary word about the “idiosyncratic” use of acronyms and initialisms within the field. (A full list is shown in Lyons et al., 2004b.) Tools, techniques, processes and methodologies are used interchangeably within the literature. However, for the benefit of this paper, the term “process” will be reserved to mean the “clinical process” under assessment for its risks, i.e.
Method
This paper aims to cluster the information in the literature to provide an initial framework to support novice users in selecting a technique for use.
The findings of a previous review of the literature (Lyons et al., 2004b) were used to structure the framework. These had been reviewed by the author – an individual with over 10 years experience in human reliability and safety. Acknowledging the findings of Everdij, 2004, Suokas, 1988 and Kirwan (1998b), this included a search using 168 generic
Results
These results aim to provide a literature-based framework towards the development of a protocol for a novice user in the healthcare sector in selecting a predictive safety technique. This framework aims to limit the choice of feasible techniques to a shortlist, so it is easier to identify a final technique. The overall protocol is shown in Fig. 1 with reference to the details of the sections required in the text.
Personnel and expertise
A number of personnel are required to carry out an analysis of any type. To be more specific, the two types of expert can be categorised as safety or human factors experts and subject matter or process experts, as defined in the sections below.
Requirements
If the resources and constraints have not sufficiently narrowed the selection, the objective of the analysis should support this goal. It may be a case that the goal is not clearly defined and there is merely and intention to understand a bit more about the errors in the process. For this, safety techniques often have common parts whereby certain techniques fulfil all or part of the requirements to a greater or lesser degree.
Knowing whether it is satisfactory to have simply the potential errors
Conclusions
Every safety expert started as a novice and usually relied on mentors to support them in learning the techniques, learning an extensive number of techniques to ensure breadth and flexibility of skill. From this point, they limit their toolbox of techniques through personal experience with (and tailoring of) the techniques, gaining expertise that is not made explicit in the literature. Healthcare professionals have limited time and support to devote to education in predictive safety analysis.
Acknowledgements
Thanks to Charles Vincent and Sally Adams and The Nuffield Trust for supporting the literature review phase of the work and to Professor P. John Clarkson and the Patient Safety Research Portfolio for supporting the later analysis. Thanks to Professor Neville Stanton and Dr Barry Kirwan for their advice.
References (89)
- et al.
The human factors in risk analyses of process plants: the control room operator model “TESEO”
Reliability Engineering
(1980) - et al.
Using Health Care Failure Mode and Effect Analysis™: The VA National Centre for Patient Safety's Prospective Risk Analysis System
Journal on Quality Improvement
(2002) - et al.
Failure modes and effects analysis application to critical care medicine
Critical Care Clinics
(2005) Representing cognitive activities and errors in HRA trees
Reliability Engineering and System Safety
(1993)- et al.
Representing context, cognition and crew performance in a shutdown risk assessment
Reliability Engineering and System Safety
(1996) - et al.
Using SHERPA to predict design-induced error on the flight deck
Aerospace Science and Technology
(2005) - et al.
Safety barrier function analysis in a process industry: a nuclear power application
International Journal of Industrial Ergonomics
(1996) Human error identification in human reliability assessment. Part 1: overview of approaches
Applied Ergonomics
(1992)Human error identification in human reliability assessment. Part 2: detailed comparison of techniques
Applied Ergonomics
(1992)The development of a nuclear chemical plant human reliability management approach: HRMS and JHEDI
Reliability Engineering and System Safety
(1997)
Human error identification techniques for risk assessment of high risk systems – Part 1: review and evaluation of techniques
Applied Ergonomics
Human error identification techniques for risk assessment of high risk systems – Part 2: towards a framework approach
Applied Ergonomics
A comparison of accident analysis techniques for safety-critical man-machine systems
International Journal of Industrial Ergonomics
Quality in healthcare: process
Best Practice & Research Clinical Anaesthesiology
System safety for robots: an energy barrier analysis
Journal of Occupational Accidents
Giving ergonomics away? The application of ergonomics methodsby novices
Applied Ergonomics
The role of safety analysis in accident prevention
Accident Analysis & Prevention
Twenty five years of HAZOPs
Journal of Loss Prevention in the Process Industries
Evaluation of accident scenarios in a Dutch steel works using a hazard and operability study
Safety Science
Analysis of errors enacted by surgical trainees during skills training courses
Surgery
Assessment of off-normal emissions from hazardous waste incinerators. Part 1:assessment of off-normal emission frequency and duration
Journal of Hazardous Materials
Use of failure modes effects analysis in improving the safety of i.v. drug administration
American Journal of Health-System Pharmacy
Issues of quality and practicability in task analysis: preliminary results from two surveys
Ergonomics
Design of a safer approach to intravenous drug infusions: failure modes effects analysis
Quality and Safety in Healthcare
Knowledge, attitudes, and practices of food service staff regarding food hygiene in Shiraz, Iran
Infection Control & Hospital Epidemiology
Apr. Prerequisite programs and food hygiene in hospitals: food safety knowledge and practices of food service staff in Ankara, Turkey
Infection Control & Hospital Epidemiology
Critical decision method: a technique for eliciting concrete assessment indicators from the intuition of NICU nurses
Advances in Nursing Science
PLFaultCAT: a product-line software fault tree analysis tool
Automated Software Engineering
Human reliability analysis for enhanced patient safety – a pilot study
Health Care Risk Report
Expert elicitation approach for performing ATHEANA quantification
Reliability Engineering and System Safety
Using cognitive task analysis to facilitate collaboration in development of simulator to accelerate surgical training
Studies in Health Technology and Informatics
FRamework Assessing Notorious Contributing Influences for Error (FRANCIE)
Human Cognitive Reliability Model for PRA Analysis
Cognitive Reliability and Error Analysis Method (CREAM)
Barrier Analysis and Accident Prevention
Human Reliability Assessors Guide
The application of HACCP in clinical engineering
Journal of Clinical Engineering
Failure in Safety-critical Systems: A Handbook of Incident and Accident Reporting
The application of Petri nets to reason about human factors problems during accident analyses
Cited by (46)
Identification and analysis of human errors in emergency department nurses using SHERPA method
2022, International Emergency NursingCitation Excerpt :Similarly, in 2015, Khandan et al. used the SHERPA method to manage errors in the field of healthcare and introduced this method as a suitable tool for identifying and evaluating human error [34]. Consequently, now that the COVID-19 pandemic has compounded healthcare issues around the world, it is highly important to identify and analyze medical errors in the hospital environment using error assessment techniques so as to establish patient safety and health planning [35–38]. Given the scope of the COVID-19 pandemic, concerns about the occurrence of human errors are increasing as such errors can detrimentally impact people’s efficiency [39].
Human reliability analysis: Exploring the intellectual structure of a research field
2020, Reliability Engineering and System SafetyA methodology for Dynamic Human Reliability Analysis in Robotic Surgery
2020, Applied ErgonomicsCitation Excerpt :HEART is one of the most well-known HRA techniques with documented applications in healthcare (Lyons et al., 2004, 2009; Williams, 1985). A closer look at the papers describing HEART applications in healthcare reveals that there is general consensus regarding the opportunity of applying HEART in healthcare, as it is a quick and useful method for analysing different healthcare processes and treatments (Castiglia et al., 2010; Chadwick and Fallon, 2012; Lyons, 2009; Lyons et al., 2004). Nevertheless, the same authors underline that the HEART technique may not be suitably developed or generic for application to all healthcare tasks, and accordingly they proposed a series of modifications to properly take into account the peculiarities of the healthcare context (Chadwick and Fallon, 2012).
Integration of multiple methods in identifying patient safety risks
2019, Safety ScienceCitation Excerpt :The only exception is that of FMEA, where the time-intensive nature of the method was often described as a significant constraint (Simsekler et al., 2019; Ward et al., 2010). Slow adoption of PHA methods has been reported previously (Lyons, 2009; Potts et al., 2014; Simsekler, 2019). It can therefore be suggested that it is essential to choose the most appropriate method for a given system, within available resources.
Human Reliability Analysis (HRA) in surgery: Identification and assessment of Influencing Factors
2018, Safety ScienceCitation Excerpt :The extant literature also reveals that the attempts for a theoretical contribution to human error analysis in healthcare mainly come from Lyons’ research group (Lyons, 2009; Lyons et al., 2004). The authors pointed out that despite the availability of several different error prediction approaches, such as Human Reliability Analysis methodologies, their current application in healthcare is still very limited (Lyons, 2009; Lyons et al., 2004). The reasons lie in the major differences between the complexity of the healthcare operations when compared to the industrial ones.
Exploring safety systems for dispensing in community pharmacies: Focusing on how staff relate to organizational components
2015, Research in Social and Administrative Pharmacy