Article Text
PDF
Abstract
Background Medical-error analyses have been conducted to determine the root cause of adverse events and near misses. More precise determination of the cause-and-effect relationship likely will require a prospective design path analysis including both direct and indirect effects.
Methods The authors performed a 6-month prospective cohort study using structural-equation modelling (SEM). Of the 879 nurses approached, 789 (89.8%) were included in the final analysis. Potential predictors provided for analysis included age, years of nursing experience, mean frequency of night shifts per month, nursing-specific job stressors, degree of depression, frequency of feeling unskilled, feeling time pressure, feeling a lack of communication between self and other hospital staff members, frequency of suffering from sleep disturbance and frequency of feeling a decrease in attention. The authors regarded a latent variable composed of frequencies for near misses and adverse events as an outcome.
Results and conclusion The SEM model constructed in this study suggested that potential root causes (exogenous variables directly or indirectly connected to the outcome which are not affected by other variables) were years of nursing experience, feeling unskilled, job stressors and sleep disturbance, with estimated standardised total (direct and indirect) effects of −0.22, 0.21, 0.008 and 0.005, respectively. A prospective design path analysis using the SEM model for both direct and indirect effects enabled a statistical exploration of root causes and estimation of their impact on the outcome. Our findings suggested such an analysis to be useful in devising countermeasures against medical errors.
- Adverse event
- near miss
- nurses
- root cause analysis
- statistics
Statistics from Altmetric.com
Introduction
Improvement in patient safety is a high-priority issue of great social import in many countries. Several studies have reported that most adverse events are due to errors of hospital staff, including nurses, and emphasised the necessity of promoting countermeasures against medical errors.1–3 Based on the results of a number of error studies, Shojania et al defined a medical error as an act or omission that leads to an unanticipated, undesirable outcome or to a substantial potential for such an outcome.4
Many medical facilities have implemented incident-reporting systems to prevent such errors, and details regarding causes of near misses and adverse events are now determined using a root cause analysis (RCA).5 6 RCA aims to determine not only the types of mistakes made but also the root cause of near misses and adverse events, thereby facilitating improvements in an organisation's activities and preventing reoccurrence of similar events.
However, the retrospective and qualitative nature of RCA is a limitation in scientific analyses examining causal relationships. When exploring the cause of an error after its occurrence, vague memory can hamper identification of a simple cause buried among daily activities, and that cause's impact on the error may therefore be underestimated.
Given that causes of medical error are often multifactorial,7 accurate estimation of a prospective root cause from among a number factors and its impact on the outcome will aid in deciding countermeasures to prevent such errors in the future. However, estimation of this root cause requires an accurate quantitative path analysis of the obtained data, for both direct and indirect effects.
In many analyses attempted thus far for the incident-reporting system, analyses using near misses as an outcome are performed separately from those using adverse events as an outcome, with each method having its own strengths and weaknesses. Because near misses and adverse events are interrelated significant outcomes, analyses based on outcomes that can be explained by both near misses and adverse events are therefore believed to be useful in developing preventive measures against medical errors.
In recent years, analysis of causal relations using structural equation modelling (SEM) has been performed with increasing frequency. SEM is a statistical technique for testing and estimating causal relations including latent variables: variables which are not measured directly, but are estimated in the model from several measured variables.8 9 Variables directly or indirectly connected to the outcome which are not affected by other variables are called exogenous variables in SEM and are equivalent to the root cause in RCA. The total effect (combination of direct and indirect effects) of each exogenous variable on outcome can be calculated in a path analysis using SEM.
A prospective design path analysis using SEM was performed in this study using the outcome of latent variables comprising near-miss and adverse-event data, with results suggesting that this method may be useful when devising countermeasures against medical errors.
Methods
Participants
This study enrolled 879 registered nurses working at Kitasato University Hospital (Kanagawa, Japan). The hospital employs a three-shift system of 8 h night work rotations. Given that senior nursing officers are in charge of staff management and not primarily involved in nursing care, they were excluded from the study. No other exclusion criteria were specified.
Study procedures and measurements
Approval to conduct the study was obtained from the Institutional Ethics Committees of Kitasato University. The objectives and procedures of the study were explained to the nursing director and senior nursing officer at the hospital, and the following information was provided to the nurses using a study information sheet: objectives and methods of the study, notice of anonymous and voluntary participation status and no penalty for non-participation, and an option to discontinue the study at any point. Informed consent was given by returning a self-reported questionnaire to the investigator.
The first self-report questionnaire enquired about the following potential predictors that have been reported to be significantly associated with medical errors10–17: gender, age, years of nursing experience, presence or absence of night shifts, frequency of night shifts, job stressors in nursing, feelings of depression, frequency of feeling unskilled, working conditions such as frequency of feeling time pressure and lack of communication among staff members, sleep disturbance and decreased attention. Job stressors in nursing were evaluated using the Nursing Stress Scale (NSS).18–20 Nurse depression was assessed using the depression scores obtained from Hospital Anxiety and Depression Scale (HADS).21 Frequencies of feeling unskilled, being under stressful work conditions, experiencing disturbed sleep and having a decreased attention were enquired about in the questionnaire as, ‘How often do you feel unskilled during work?,’ ‘How often do you feel strong time pressure during work?,’ ‘How often do you feel a lack of communication between yourself and other hospital staff members?,’ ‘How often do you suffer from sleep disturbance?’ and ‘How often do you feel decreased attention during work?’ Response choices were ‘no,’ ‘occasionally,’ ‘frequently’ or ‘always.’
The second questionnaire, distributed 6 months after the first questionnaire, enquired about the frequency of self-perceived near misses and adverse events experienced in the previous 6 months. The participants had been instructed ahead of time to assess and record the number of near misses and adverse events during the observation period. All reported questionnaires were sealed in individual envelopes before collection. To link the first and second surveys, participants were instructed in the initial questionnaire to provide a self-defined four-digit ID number and a hint to be enquired about, should the ID number be forgotten, such as the name of the participant's junior high school.
Definition of ‘near miss’, ‘adverse event’ and ‘medical error risk’
An unanticipated and undesirable outcome caused by medical errors is often categorised as either a ‘near miss’ (close misses or potential adverse events) or an ‘adverse event’ (accidents or harmful events).5 A near miss caused by error was defined as an unanticipated incident in which an error was made but no harm occurred.13 22 To assess the frequency of perceived near misses in the previous 6 months, the questionnaire enquired, ‘In the previous 6 months, how often did you experience a self-error-related near miss in which an error was made, but no harm occurred to a patient?’ An adverse event caused by an error was defined as an unanticipated incident in which a self-error was made, and harm occurred to a patient,23 and the questionnaire enquired about the frequency of adverse events by, ‘In the previous 6 months, how often did you inflict a self-error-related harm on a patient or disadvantage a patient, regardless of event severity?’
In SEM, both the observed variables and latent variables composed of observed variables can be input into the model.24 We referred to the concept of risk of medical errors hidden by near misses and adverse events as ‘medical error risk,’ which is a latent variable composed of frequencies of near misses and adverse events as well as other factors (error variables) in SEM. Latent variables (hidden variables or hypothetical constructs) are those not directly observed but rather inferred (through a mathematical model) from other observed variables (eg, intelligence, confidence).8 9
Statistical analyses
A bivariate analysis was initially performed to determine the variables for our SEM model. Correlations of observed variables with frequencies of near misses and adverse events were analysed for continuous variables, whereas a t test was performed for categorical variables such as gender and ‘working the night shift’ to compare differences in occurrences of near misses and adverse events by gender and working the night shift. Only variables which showed a statistically significant relationship with near misses or adverse events in the above univariate analysis were input into SEM to obtain the simplest and most suitable model.
Second, we constructed several models based on the hypothesis that decreased attention is an important mediator of medical errors. A goodness-of-fit index (GFI), adjusted GFI (AGFI), comparative fit index (CFI) and root mean squared error of approximation (RMSEA) were used for goodness-of-fit indices. The criteria used were GFI ≥0.95, AGFI ≥0.90, CFI ≥0.90 and RMSEA ≤0.08.25 26 Of those models which showed sufficient compatibility, the model with the lowest Akaike information criterion (AIC) value was adopted as the final model. SPSS V.16.0 and AMOS version 7.0 (SPSS) were used for analyses, and statistical significance was set at 0.05, with all tests two-tailed.
Results
Baseline self-report questionnaires were obtained from 831 of 879 nurses (94.5%), with no information available regarding the 48 nurses who did not respond to the baseline questionnaire. Of these 831, 819 (93.2%) responded to the second questionnaire. Of these 819 who responded to both questionnaires, the 789 (89.8%) who answered the questions completely were included in statistical analyses. No significant differences were noted in gender, age, years of nursing experience, frequency of night shifts, NSS score, depression score or working condition score between the 42 nurses who responded only to the baseline questionnaire or answered questions incompletely.
The demographic characteristics of the study nurses are shown in table 1. Of the 789 nurses included in the final analysis, 731 (93.49%) were female, with a mean (SD) age of 28.7 (6.3) years and mean (SD) of 6.8 (5.9) years of experience. A total of 748 nurses (94.8%) worked on a rotation schedule including a nightshift. Further, of the 789 included in the final analysis, 739 (93.7%) experienced near misses during the 6-month observation period, with a mean frequency (SD) of 4.0 (4.3) for near misses in 6 months. Of these 789, 451 (57.2%) experienced adverse events during the 6-month observation period, with a mean frequency (SD) of 1.2 (1.5).
Demographic characteristics of study participants (N=789)
Table 2 shows the Pearson correlation coefficient matrix of the observed variables. Except for frequencies of night shifts and experiencing time pressure, predictor variables were significantly correlated with either or both the frequencies of near misses and adverse events. The mean frequencies (SD) of near misses and adverse events in men and women were 3.9 (4.1) versus 4.3 (6.1), p=0.08, and 1.2 (1.5) versus 1.1(1.4), p=0.36, respectively, with no significant difference observed. In contrast, the mean frequencies (SD) of near misses and adverse events with or without night shifts were 5.9 (6.0) versus 3.9 (4.2), p<0.001, and 2.2 (2.0) versus 1.2(1.5), p=0.03, respectively, with the mean frequency of near misses and adverse events found to be significantly higher in nurses who worked night shifts compared with those who did not.
Pearson correlation coefficients matrix of the measured variables
The following predictors were used to construct the most appropriate SEM after excluding gender, frequency of night shifts and experiencing time pressure on the basis of the univariate analyses above: age, years of nursing experience, presence or absence of night shifts, nursing-specific job stressors using NSS, degree of depression, frequency of feeling unskilled, feeling a lack of communication among medical staff members, frequency of suffering from sleep disturbance and frequency of feeling a decrease in attention.
The final model with significant pathways is illustrated in figure 1. In this model, all of the statistically significant paths (p<0.05) and correlations (p<0.05) are expressed, except for those related with error variables. The path coefficients are all standardised. To avoid making the figure overly complex, descriptions of error coefficients are omitted. Sufficient model compatibility was indicated with the following scores: GFI=0.98, AGFI=0.96, CFI=0.95, RMSEA=0.05. The AIC value was the lowest in this model (138.6).
Association between medical error risk and predictors in the final structural-equation model. e, error variable.
Model compatibility was unsatisfactory for the model containing both age and years of nursing experience. Just as with other multivariable analyses, compatibility may be reduced in SEM when predictors that are too strongly correlated are used in the same model. Similarly, in the present study, the correlation between years of nursing experience and feeling unskilled, which significantly elevated the compatibility of the entire model, was weakened when the correlation between both variables was input into the model due to an extremely strong correlation between age and years of nursing experience (correlation coefficient=0.9), which resulted in reduced compatibility for the entire model. As such, we opted to ultimately include only one or the other in our model. While compatibility was almost identical between the model using ‘age’ and that using ‘years of nursing experience’, AIC was slightly higher in the model using ‘age’ (145.3). We therefore decided to use ‘years of nursing experience’ in the final model.
In the final model, factors directly affecting risk of medical error were years of nursing experience (path coefficient: −0.22), frequency of feeling unskilled (path coefficient: 0.13) and frequency of decreased attention (path coefficient: 0.25). ‘Years of nursing experience’ was negatively and significantly correlated with job stressors, and frequency of feeling unskilled, and directly affected risk of medical error. Frequency of feeling unskilled was correlated with years of nursing experience, job stressors and frequency of sensing a lack of communication, and directly affected medical error risk. In addition, this factor indirectly affected mediator error risk via depression and decreased attention. Decreased attention was shown to be a mediator affected by feeling unskilled and depressed, and depression was shown to be affected by job stressors and sleep disturbance.
Findings from the present study suggested that exogenous variables directly or indirectly connected to the outcome which were not affected by other variables in the present SEM model were years of nursing experience, feeling unskilled, job stressors and sleep disturbance. The estimated standardised total (direct and indirect) effects of exogenous variables such as years of nursing experience, feeling unskilled, job stressors and sleep disturbance on risk of medical error were −0.22, 0.21, 0.008 and 0.005, respectively.
Discussion
One of the characteristics of this study was its prospective design. A prospective design allows for systematic collection of information on prospective factors in advance and assurance of temporality (meaning that the effect must occur after the cause), both essential principles for verifying a causal relationship. When investigating the cause of an error after it occurs, as is done in many RCA, information on potential causes may lose accuracy due to vagueness of memory, overly impressive events may be preferentially reported as necessary information, and primary elements may be overshadowed by daily activities or common system elements, thereby escaping notice and being underestimated. While a prospective study design may not be suitable when making emergency decisions on countermeasures against medical errors, this design will prove useful when considering such measures in an evidence-based manner from mid- and long-term perspectives.
A second characteristic of this study was the construction of an SEM with high validity using model compatibility as an index, thereby enabling estimation of a root cause from among various candidate causes and calculation of total effect for estimating the impact of a root cause on outcome. Findings from the present study suggested that having fewer years of experience, frequently feeling unskilled and having decreased attention all directly affected medical error risk. In our final model, years of nursing experience and frequency of feeling unskilled were exogenous variables, and decreased attention was shown to be a mediator affected by the frequency of feeling unskilled and depression, as we anticipated. Depression was connected to the frequency of feeling unskilled and two other exogenous variables: job stressors and sleep disturbance. Taken together, our findings thus indicated that the exogenous variables directly or indirectly related to medical error risk were relatively few years of experience, frequently feeling unskilled, and presence of job stressors and sleep disturbance. Therefore, in the model constructed in this study, these four variables were reasonably speculated to be root causes. In other studies performed thus far as well, these four causes were reported to be particularly important for elucidating causes of medical errors.10 16 27 28
In the SEM model, the total effect of predictive variables on the outcome can be calculated based on coefficient values of direct or indirect paths. Among the exogenous variables cited in the present model, standardised total effects of years of nursing experience and increased frequency of feeling unskilled to medical error risk were much higher than those for other variables, suggesting that the primary method of preventing risk of medical error is to provide as many feasible measures as possible to novice nurses who have little nursing experience and feel unskilled, such as constant availability of expertise given workload unpredictability, a social climate which accounts for expectations of novice performers, realistic expectations of novice decision-making ability during complex situations and strategies to recognise and intervene when novices are at risk for error.29
A third characteristic of this study was defining the concept comprising the frequency of near misses and adverse events in our SEM model as ‘medical error risk’ and handling it as an outcome. Although adverse-event analysis can identify factors directly related to adverse events, the data tend to be insufficient due to low frequency and resistance to event reporting. In recent years, however, more analyses have been conducted regarding near misses as outcomes.4 30 31 The abundant data available and lower resistance to data collection than adverse events allow for quantitative analysis of near misses.32 33 However, no firm conclusion has been drawn regarding whether using adverse events or near misses as outcomes is more effective at preventing medical errors. At present, both types of analyses are considered valuable, and therefore analyses based on outcomes that can be explained by both near misses and adverse events are believed to be useful in developing preventive measures against medical errors. An additional investigation of compatibility of several models using the frequencies of near miss or adverse events as outcomes showed that compatibility was always lower in such a model compared with compatibility in a model using latent variables explained both by frequencies of near misses and adverse events as an outcome (data not shown). Outcome selection is important when analysing data for prevention of medical errors, and using a comprehensive concept comprising the frequency of both near misses and adverse events as an outcome was considered more effective than selecting only one of these two concepts.
Several limitations to generalisation of results from our study warrant mention. First, predictive factors in this study do not include all of the factors reported thus far. While a great number of factors are thought to be related to incidence of medical errors, checking so many factors was difficult due to survey limitations. The factors in this study were selected because they are highly likely to be important causes of medical error, according to our judgement. A substantially different model might be attained, should other significant factors be used for input. Second, the numbers of near misses and adverse events were not counted by interview or objective observation. Further, to maintain confidentiality during this study, we were unable to use information submitted to the incident-reporting system at the present hospital. Therefore, some measurement and recall bias may have occurred. Although participants were instructed to check and record the number of near misses and adverse events once a month during the observation period, a 6-month period was found to be too long for participants to recall the number of near misses accurately. Third, participants were nurses working in teaching hospitals and had younger mean ages than most other hospitals, as is common in Japan. Further, some nurses performed additional jobs unrelated to the routine jobs of nurses working in a general hospital, possibly contributing to selection bias.
Conclusion
In the present study, we analysed causes of medical error using a method other than RCA (involving retrospective and qualitative analysis of errors after occurrence), which is frequently used to investigate causes of medical errors. In our method, prospective causes of medical errors are investigated in advance, and occurrences of near miss and adverse events are prospectively followed up (not in a study; the incident-reporting system is typically used in a clinical setting). A latent variable termed ‘medical error risk,’ which is commonly hidden behind near misses and adverse events, is handled as an outcome, and a prospective design path analysis including both direct and indirect effects using SEM is performed to quantitatively estimate the impact of the root causes on ‘medical error risk’. Incorporation of this analysis method in the current incident-reporting system would be useful when determining countermeasures against medical error from mid- and long-term perspectives. To this end, we suggest that the following measures be implemented: factors that might cause medical errors should be surveyed regularly and continuously, and after a sufficient number of cases of near misses and adverse events have accumulated under the current incident-reporting system, root causes for outcome of latent variables that can be explained by both near misses and adverse events should be analysed using SEM. Prioritised countermeasures should then be formulated, based on the value of the total effect by the estimated risk factor.
We believe that patient safety can be improved by incorporating countermeasures formulated based on the prospective and quantitative analysis method as proposed in this study, in addition to the current measures developed based on the quantitative analysis method.
References
Footnotes
Funding Grant-in-Aid (No 16591159) from the Japan Society for the Promotion of Science.
Competing interests None.
Ethics approval Ethics approval was provided by the Institutional Ethics Committees of Kitasato University.
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Corrections