Elsevier

Applied Ergonomics

Volume 40, Issue 3, May 2009, Pages 379-395
Applied Ergonomics

Towards a framework to select techniques for error prediction: Supporting novice users in the healthcare sector

https://doi.org/10.1016/j.apergo.2008.11.004Get rights and content

Abstract

Whilst healthcare has increased its awareness of the retrospective safety assessment techniques, such as root cause analysis, adoption of the corresponding predictive safety assessment techniques has been slow and sporadic. Reasons for this may include lack of support in technique choice and practical knowledge in the published literature. Whilst there have been many publications on these techniques, few have aimed to support the novice user in selecting a technique from the broad array of choice to facilitate targeting of education in techniques for specific purposes.

This paper aims to address this through collecting an evidence base towards developing a bottom-up (resources and constraints) and top-down (requirements) approach to technique selection.

Conclusions indicate there is a lack of practical experiences described in the literature to conclusively define a technique for selection and a need for a dedicated research in this area to make it accessible for healthcare and other novice users.

Introduction

Incident investigation techniques have been increasingly used in healthcare and are supported by literature specifically written to help novices in choosing techniques (Johnson, 2003). However, a similar pattern has not occurred for predictive safety techniques. Despite many decades of acceptance of the predictive safety techniques in other industries, Lyons et al. (2004a) found only seven techniques had been published as being used for healthcare application (change analysis, FMEA, HAZOP, influence diagrams, SHERPA, event trees and fault trees). This was particularly noteworthy when task analysis – the precursory step for many human reliability analysis techniques – has been applied to several areas within healthcare.

The reasons for its narrow application may be the lack of awareness that there are so many usable techniques or due to the challenge of choosing between the overwhelming number of techniques – with 520 safety assessment methodologies identified for supporting air traffic management (Everdij, 2004). Although awareness and understanding of the practical application of such a great number of techniques may appear impossible, it should be emphasised that not all the techniques are discrete, with many variants evolving for a subset of the techniques. One idiosyncrasy of safety assessment techniques is that there is a popular trend to give the techniques acronymic (HEART) or initialistic (HTA) names. Therefore, often techniques that are identical in form have been given different names due to application in different domains or have minor changes made by authors. Conversely identical techniques have evolved with slightly different names – e.g. Safety barrier function analysis (Kecklund et al., 1996), accident evolution barrier analysis (Svenson, 1991, Svenson, 2001), energy barrier analysis (Rahimi, 1986) and barrier analysis (Hollnagel, 2004). Kirwan (1998a) outlines an approximate evolution relationship of many of these techniques (e.g. FMEA, HAZOP and event tree based techniques) including cross-links between these.

Even in industries more familiar with reliability engineering techniques, it has been speculated that these techniques were used so scarcely due to the unavailability of the means and/or techniques, technique complexity (Paz Barroso and Wilson, 2000) or lack of information on resources required (Ainsworth and Marshall, 1998). Pradhan et al. (2001) suggest the cultural norms of healthcare also add to the challenges – perhaps resulting in demands for tailor-made instead of the industry standard techniques.

The continuous professional development required by healthcare professionals is broader than the remits of safety assessment. Even with governmental support, this may limit opportunities to learn a range of safety techniques. It may be preferable for healthcare users unfamiliar with the field to choose a technique to solve a specific problem and then target the technique education accordingly.

Otherwise, there is a danger that novices are using techniques to guide safety-related decisions without the training appropriate to gain the validity and reliability assumed from the technique (Stanton and Young, 2003).

Whilst there have been a number of reviews of the human reliability analysis and similar techniques (Everdij, 2004; Humphreys, 1988; Suokas, 1988, Kirwan, 1992a, Kirwan, 1992b, Kirwan, 1994, 1998; Stanton et al., 2005, Wreathall and Nemeth, 2004), most have focussed more on the accuracy and reliability of the techniques rather than the requirements of the problem to be analysed. Together with the issues of a lack of technique awareness and expertise, the pressures on healthcare personnel will tend to make healthcare professionals more reluctant to invest in the rigorous education requirements for the more complex techniques, designed to produce more accurate results. This may lead to personnel choosing the “quick and dirty” techniques even if they are not the most appropriate for the problem. Furthermore, the previous reviews have always been written in a technique-by-technique format thus demanding time commitment in investigating techniques that would not be chosen, simply to identify the technique that best matches the features of the chosen problem. In many cases, this assumes the readers to have some knowledge or experience in the field and already have a mental model of many or all the techniques. In short, what is required is a user-focussed approach for selecting a technique – namely a bottom-up method of selection based on the resources, constraints and requirements of the user.

As yet, there has been no dedicated study to research the usability of the techniques, the limits of requirements for each technique in terms of time or human resources or any efforts to support the novice in selecting a technique to analyse their problem. Therefore, this paper provides a first step in providing a literature-based framework for selection of techniques.

Section snippets

Terms and definitions

Sources of confusion are the terms and definitions used when describing the techniques. Therefore this section provides a cautionary word about the “idiosyncratic” use of acronyms and initialisms within the field. (A full list is shown in Lyons et al., 2004b.) Tools, techniques, processes and methodologies are used interchangeably within the literature. However, for the benefit of this paper, the term “process” will be reserved to mean the “clinical process” under assessment for its risks, i.e.

Method

This paper aims to cluster the information in the literature to provide an initial framework to support novice users in selecting a technique for use.

The findings of a previous review of the literature (Lyons et al., 2004b) were used to structure the framework. These had been reviewed by the author – an individual with over 10 years experience in human reliability and safety. Acknowledging the findings of Everdij, 2004, Suokas, 1988 and Kirwan (1998b), this included a search using 168 generic

Results

These results aim to provide a literature-based framework towards the development of a protocol for a novice user in the healthcare sector in selecting a predictive safety technique. This framework aims to limit the choice of feasible techniques to a shortlist, so it is easier to identify a final technique. The overall protocol is shown in Fig. 1 with reference to the details of the sections required in the text.

Personnel and expertise

A number of personnel are required to carry out an analysis of any type. To be more specific, the two types of expert can be categorised as safety or human factors experts and subject matter or process experts, as defined in the sections below.

Requirements

If the resources and constraints have not sufficiently narrowed the selection, the objective of the analysis should support this goal. It may be a case that the goal is not clearly defined and there is merely and intention to understand a bit more about the errors in the process. For this, safety techniques often have common parts whereby certain techniques fulfil all or part of the requirements to a greater or lesser degree.

Knowing whether it is satisfactory to have simply the potential errors

Conclusions

Every safety expert started as a novice and usually relied on mentors to support them in learning the techniques, learning an extensive number of techniques to ensure breadth and flexibility of skill. From this point, they limit their toolbox of techniques through personal experience with (and tailoring of) the techniques, gaining expertise that is not made explicit in the literature. Healthcare professionals have limited time and support to devote to education in predictive safety analysis.

Acknowledgements

Thanks to Charles Vincent and Sally Adams and The Nuffield Trust for supporting the literature review phase of the work and to Professor P. John Clarkson and the Patient Safety Research Portfolio for supporting the later analysis. Thanks to Professor Neville Stanton and Dr Barry Kirwan for their advice.

References (89)

  • B. Kirwan

    Human error identification techniques for risk assessment of high risk systems – Part 1: review and evaluation of techniques

    Applied Ergonomics

    (1998)
  • B. Kirwan

    Human error identification techniques for risk assessment of high risk systems – Part 2: towards a framework approach

    Applied Ergonomics

    (1998)
  • T. Kontogiannis et al.

    A comparison of accident analysis techniques for safety-critical man-machine systems

    International Journal of Industrial Ergonomics

    (2000)
  • M. Pradhan et al.

    Quality in healthcare: process

    Best Practice & Research Clinical Anaesthesiology

    (2001)
  • M. Rahimi

    System safety for robots: an energy barrier analysis

    Journal of Occupational Accidents

    (1986)
  • N.A. Stanton et al.

    Giving ergonomics away? The application of ergonomics methodsby novices

    Applied Ergonomics

    (2003)
  • J. Suokas

    The role of safety analysis in accident prevention

    Accident Analysis & Prevention

    (1988)
  • C.D. Swann et al.

    Twenty five years of HAZOPs

    Journal of Loss Prevention in the Process Industries

    (1995)
  • P. Swuste et al.

    Evaluation of accident scenarios in a Dutch steel works using a hazard and operability study

    Safety Science

    (1997)
  • B. Tang et al.

    Analysis of errors enacted by surgical trainees during skills training courses

    Surgery

    (2005 Jul)
  • Y. Zeng et al.

    Assessment of off-normal emissions from hazardous waste incinerators. Part 1:assessment of off-normal emission frequency and duration

    Journal of Hazardous Materials

    (1991)
  • W. Adachi et al.

    Use of failure modes effects analysis in improving the safety of i.v. drug administration

    American Journal of Health-System Pharmacy

    (2005)
  • L. Ainsworth et al.

    Issues of quality and practicability in task analysis: preliminary results from two surveys

    Ergonomics

    (1998)
  • M. Apkon et al.

    Design of a safer approach to intravenous drug infusions: failure modes effects analysis

    Quality and Safety in Healthcare

    (2004)
  • M. Askarian et al.

    Knowledge, attitudes, and practices of food service staff regarding food hygiene in Shiraz, Iran

    Infection Control & Hospital Epidemiology

    (2004 Jan)
  • M. Bas et al.

    Apr. Prerequisite programs and food hygiene in hospitals: food safety knowledge and practices of food service staff in Ankara, Turkey

    Infection Control & Hospital Epidemiology

    (2005)
  • B. Crandall et al.

    Critical decision method: a technique for eliciting concrete assessment indicators from the intuition of NICU nurses

    Advances in Nursing Science

    (1993 Sep)
  • J. Dehlinger et al.

    PLFaultCAT: a product-line software fault tree analysis tool

    Automated Software Engineering

    (2006 Jan)
  • Everdij, M. 2004. Review of Techniques to Support the EATMP Safety Assessment Methodology, EEC Note 01/04. Volume II:...
  • K. Farmery et al.

    Human reliability analysis for enhanced patient safety – a pilot study

    Health Care Risk Report

    (2006 Mar.)
  • J. Forester et al.

    Expert elicitation approach for performing ATHEANA quantification

    Reliability Engineering and System Safety

    (2004)
  • T. Grunwald et al.

    Using cognitive task analysis to facilitate collaboration in development of simulator to accelerate surgical training

    Studies in Health Technology and Informatics

    (2004)
  • Hahn, H.A., deVries, J.A. 1991. Identification of human errors of commission using Sneak Analysis. In: Proceedings of...
  • Hahn, B.B., Rall, E., Klinger, D.W. 2003. Cognitive Task Analysis of the Warning Forecaster Task. Final Report. The...
  • L.N. Haney

    FRamework Assessing Notorious Contributing Influences for Error (FRANCIE)

    (1999 Jul.)
  • G.W. Hannaman et al.

    Human Cognitive Reliability Model for PRA Analysis

    (1984)
  • E. Hollnagel

    Cognitive Reliability and Error Analysis Method (CREAM)

    (1998)
  • E. Hollnagel

    Barrier Analysis and Accident Prevention

    (2004)
  • P. Humphreys

    Human Reliability Assessors Guide

    (1988 Oct.)
  • W.A. Hyman

    The application of HACCP in clinical engineering

    Journal of Clinical Engineering

    (2003 Jul/Aug/Sep.)
  • C.W. Johnson

    Failure in Safety-critical Systems: A Handbook of Incident and Accident Reporting

    (2003)
  • C.W. Johnson

    The application of Petri nets to reason about human factors problems during accident analyses

  • Kallus, K.W., Barbarino, M., Van Damme, D.1998. Integrated Task and Job Analysis of Air Traffic Controllers. Phase 1....
  • Kennedy, R.J., Slater, K. 2005, Airspace Complexity and Safety Analysis Hazard and Operability (HAZOP). EUROCONTROL...
  • Cited by (46)

    • Identification and analysis of human errors in emergency department nurses using SHERPA method

      2022, International Emergency Nursing
      Citation Excerpt :

      Similarly, in 2015, Khandan et al. used the SHERPA method to manage errors in the field of healthcare and introduced this method as a suitable tool for identifying and evaluating human error [34]. Consequently, now that the COVID-19 pandemic has compounded healthcare issues around the world, it is highly important to identify and analyze medical errors in the hospital environment using error assessment techniques so as to establish patient safety and health planning [35–38]. Given the scope of the COVID-19 pandemic, concerns about the occurrence of human errors are increasing as such errors can detrimentally impact people’s efficiency [39].

    • A methodology for Dynamic Human Reliability Analysis in Robotic Surgery

      2020, Applied Ergonomics
      Citation Excerpt :

      HEART is one of the most well-known HRA techniques with documented applications in healthcare (Lyons et al., 2004, 2009; Williams, 1985). A closer look at the papers describing HEART applications in healthcare reveals that there is general consensus regarding the opportunity of applying HEART in healthcare, as it is a quick and useful method for analysing different healthcare processes and treatments (Castiglia et al., 2010; Chadwick and Fallon, 2012; Lyons, 2009; Lyons et al., 2004). Nevertheless, the same authors underline that the HEART technique may not be suitably developed or generic for application to all healthcare tasks, and accordingly they proposed a series of modifications to properly take into account the peculiarities of the healthcare context (Chadwick and Fallon, 2012).

    • Integration of multiple methods in identifying patient safety risks

      2019, Safety Science
      Citation Excerpt :

      The only exception is that of FMEA, where the time-intensive nature of the method was often described as a significant constraint (Simsekler et al., 2019; Ward et al., 2010). Slow adoption of PHA methods has been reported previously (Lyons, 2009; Potts et al., 2014; Simsekler, 2019). It can therefore be suggested that it is essential to choose the most appropriate method for a given system, within available resources.

    • Human Reliability Analysis (HRA) in surgery: Identification and assessment of Influencing Factors

      2018, Safety Science
      Citation Excerpt :

      The extant literature also reveals that the attempts for a theoretical contribution to human error analysis in healthcare mainly come from Lyons’ research group (Lyons, 2009; Lyons et al., 2004). The authors pointed out that despite the availability of several different error prediction approaches, such as Human Reliability Analysis methodologies, their current application in healthcare is still very limited (Lyons, 2009; Lyons et al., 2004). The reasons lie in the major differences between the complexity of the healthcare operations when compared to the industrial ones.

    View all citing articles on Scopus
    View full text