Article Text
Abstract
Background Burnout is widespread among healthcare providers and is associated with adverse safety behaviours, operational and clinical outcomes. Little is known with regard to the explanatory links between burnout and these adverse outcomes.
Objectives (1) Test the psychometric properties of a brief four-item burnout scale, (2) Provide neonatal intensive care unit (NICU) burnout and resilience benchmarking data across different units and caregiver types, (3) Examine the relationships between caregiver burnout and patient safety culture.
Research design Cross-sectional survey study.
Subjects Nurses, nurse practitioners, respiratory care providers and physicians in 44 NICUs.
Measures Caregiver assessments of burnout and safety culture.
Results Of 3294 administered surveys, 2073 were returned for an overall response rate of 62.9%. The percentage of respondents in each NICU reporting burnout ranged from 7.5% to 54.4% (mean=25.9%, SD=10.8). The four-item burnout scale was reliable (α=0.85) and appropriate for aggregation (intra-class correlation coefficient-2=0.95). Burnout varied significantly between NICUs, p<0.0001, but was less prevalent in physicians (mean=15.1%, SD=19.6) compared with non-physicians (mean=26.9%, SD=11.4, p=0.0004). NICUs with more burnout had lower teamwork climate (r=−0.48, p=0.001), safety climate (r=−0.40, p=0.01), job satisfaction (r=−0.64, p<0.0001), perceptions of management (r=−0.50, p=0.0006) and working conditions (r=−0.45, p=0.002).
Conclusions NICU caregiver burnout appears to have ‘climate-like’ features, is prevalent, and associated with lower perceptions of patient safety culture.
- Safety culture
- Health services research
- Healthcare quality improvement
Statistics from Altmetric.com
Introduction
Burnout describes a process beginning with high and sustained levels of stress resulting in feelings of irritability, fatigue, detachment and cynicism.1 In service professions, stress originates from frequent intense interactions with clients with complex problems.2 These high demands, combined with lack of support, result in burned-out employees.3 Hallmark features of burnout include a combination of emotional exhaustion, depersonalisation and a reduced sense of personal accomplishment.4
In healthcare, various causes of burnout have been described, and include chronic stress from working with patients suffering from complex physical, psychological and social problems2 ,4; unsupportive or inadequate work environments that lack support for following traumatic events; conflict with colleagues; and long or irregular shifts.5 Healthcare workers in the neonatal intensive care unit (NICU) setting may particularly struggle to balance work and personal lives amidst an onslaught of new rules and technologies, as well as high expectations for the seamless delivery of empathic, high-quality care.6 ,7
Burnout is pervasive throughout healthcare, with one out of three nurses and physicians meeting criteria.8 ,9 Reports of the prevalence of burnout among groups of healthcare workers vary widely, ranging from 27% to 86%.9–11 Burnout is of particular concern to healthcare because of adverse effects on the quality of patient care and potentially tragic consequences for patients, especially fragile preterm infants.12 ,13
We define resilience here as a combination of abilities and characteristics that interact dynamically to allow an individual to bounce back, cope successfully and function above the norm in spite of significant stress or adversity. Although there are many valid constructs that interface with the concept of resilience, that is, burnout, depression, subjective happiness and work-life balance, there is one domain in particular that is widely used, well-understood, and linked to important clinical and operational outcomes: the emotional exhaustion subscale from the Maslach Burnout Inventory.4 ,13 ,14 Emotional exhaustion is used to assess innovation fatigue and feelings of detachment and frustration with work—the polar opposite of resilience being the ‘ability to cope’.
Little is known regarding the prevalence of burnout and resilience among NICU staff and the pathways through which burnout adversely affects care quality and safety are only beginning to be understood. In this study we attempt to shed light on these relations. Our objectives were to:
-
Test the psychometric properties of a brief four-item burnout scale,
-
Provide NICU burnout and resilience benchmarking data across different units and caregiver types, and to
-
Examine the relationships between caregiver burnout and patient safety culture.
Methods
Selection of NICUs
This cross-sectional survey study was performed among a voluntary sample of NICUs participating in two simultaneous quality improvement initiatives organised by the California Perinatal Quality Care Collaborative focused on Delivery Room Management.15
For the current study, we assembled a survey to investigate burnout and safety culture using existing validated metrics from several instruments (detailed below). We offered to analyse and feedback a survey of safety culture and workforce engagement to all 61 NICUs who participated in the improvement initiative, 44 of which accepted. The survey was administered at the onset of the improvement initiative (between June and September 2011).
Staff with a 0.5 full time equivalent (FTE) or greater time commitment to the NICU for at least the four consecutive weeks prior to survey administration was invited to participate. Paper-based surveys were administered during routine departmental and staff meetings. Surveys were returned to a locked box or sealable envelope to maintain confidentiality. Individuals not present in routine meetings were hand delivered a survey, pencil and return envelope. This administration technique has generated high response rates.16 ,17 Administration of the survey was executed by California Perinatal Quality Care Collaborative and a de-identified data set was transmitted to Dr Profit for analysis. The study was approved by the Institutional Review Boards at Stanford University and Baylor College of Medicine.
Measures
The entire survey can be accessed in the web appendix. Measures relevant for this paper were part of a survey on safety culture and organisational determinants of quality. These included select items of the Safety Attitudes Questionnaire (SAQ),16 ,18 and the Maslach Burnout Inventory.19
Safety culture
Of the several safety culture survey instruments in the literature, the SAQ is widely used, has good psychometric properties,16 and is associated with clinical outcomes.20–24 The SAQ contains 30 items that load on six domains: teamwork climate, safety climate, job satisfaction, perceptions of management, stress recognition and working conditions. Each item is rated on a 5-point Likert scale ranging from 1 (disagree strongly) to 5 (agree strongly). The SAQ also captures respondent characteristics including job position, years in specialty, primary work area (paediatric, adult or both), gender and predominant work shift. Job positions included attending physicians (MDs), fellow MDs, neonatal nurse practitioners (NNPs), registered nurse (RNs), respiratory care providers and others.
Safety culture scale scores at the NICU level were calculated using the standard method: as the per cent of respondents within a NICU that had a mean score across all scale items of ‘slightly agree’ or ‘strongly agree’. For this purpose individual responses are transformed onto a 0–100-point scale score according to the following formula:
In order to calculate the per cent of respondents who are positive (ie, per cent agreement), one calculates the per cent of respondents within a NICU who received a scale score of 75 or higher.16 ,17 We call this ‘percentage agree’ or ‘percentage reporting good ‘enter scale name (safety, teamwork, etc)’’.
Burnout
To assess burnout, we used an abbreviated four-item Emotional Exhaustion scale, which is based on the Maslach Burnout Inventory,19 and which we have shown to be reliable and valid in other settings.25 This scale was adapted to the survey format of the SAQ, which changed its response scale and scoring. Therefore, psychometric exploration of this revised scale became necessary. The response scale ranged from 1 (disagree strongly) to 5 (agree strongly).
Burnout was computed by taking the mean of the four items, transforming them to a 0–100-point scale score according to the following formula:
In order to calculate the per cent of respondents who are burned out (ie, per cent that agree with burnout items), one calculates the per cent of respondents within a NICU who received a scale score of 50 or higher. The 50% threshold groups ‘neutral’ responses together with ‘agree’ responses as previously described in the literature. We call this the ‘percentage reporting burnout’. We have used a similar approach with safety culture assessments and found the metric to be meaningful to providers.16 ,18 ,26
We set a high bar for resilience. Resilience was defined as individuals who had an average burnout score (out of 0–100) of less than or equal to 25 (ie, on average rated the items in the range of strongly disagree). Within each NICU we obtained the frequency of respondents with a score of 25 or lower and call this ‘percentage reporting resilience.’ Note that individuals who had an average burnout score (out of 0–100) between 26 and 49 were not included in either the burnout or the resilience groups. Therefore, with a given NICU, the sum of the percentage burned out and the percentage who are resilient will not necessarily equal 100.
Analyses
Objective 1—Test the psychometric properties of the burnout scale in the NICU setting
We used reliability analyses to evaluate the four-item emotional exhaustion scale. Internal consistency reliability was assessed overall and by job position using Cronbach's coefficient α. To verify the single factor structure of the emotional exhaustion scale, we performed a multilevel confirmatory factor analysis using geomin oblique rotation and maximum likelihood estimation to account for the nesting of individual caregivers within NICUs. The multilevel confirmatory factor analysis corrects the between-group covariance matrix so that an unbiased between-group factor structure is obtained.27 To assess goodness of fit, the comparative fit index (CFI),28 the root-mean square error of approximation (RMSEA),29 and the within-NICU and between-NICU standardised root mean square residual (SRMR) were examined. According to Kline,30 CFI values greater than 0.90 reflect good model fit. For the RMSEA and SRMR, values below 0.05 indicate close fit, values around 0.08 indicate adequate fit and values above 0.10 indicate poor fit.27 The χ2 is reported (with significant values indicating poor fit); however, it is considered a more useful means of comparing nested models than an absolute indicator of model fit because it may be significant even when all other fit indices illustrate adequate fit.31
A basic criterion required to adequately assess climate constructs is that individual perceptions show high agreement within units (eg, NICUs) and high variance between units. Burnout is conceptualised at the NICU level of analysis, so we calculated intraclass correlation coefficients (ICCs) to justify aggregation of caregivers within their NICUs. The ICC-1 statistic is a measure of between-group variability and the ICC-2 statistic is a measure of the reliability of the group means. To calculate ICC-1 and ICC-2, a one-way analysis of variance is conducted on the individual level responses, with NICU as the independent variable. ICC-1 can be interpreted as the proportion of total variance that is explained by unit membership with values ranging from −1 to +1 and values between 0.05 and 0.30 being most typical. Whereas ICC-1 provides an estimate of the reliability of a single RN’s assessment of the unit mean, ICC-2 provides an overall estimate of the reliability of unit means. The closer ICC-2 is to 1.00, the more reliably NICUs can be distinguished based on individual respondents’ perceptions of burnout with values equal to or above 0.70 being acceptable.16 ,32
Objective 2—Provide burnout and resilience benchmarking data for NICUs and caregiver types
We used descriptive analyses such as frequencies, percentages, means (±SD) and graphs to describe demographics and our three variables: caregiver burnout, safety climate and teamwork climate. To test for differences in burnout, we focused on per cent reporting burnout by NICU and by caregiver type. A between-groups analysis of variance was used to examine differences in burnout by NICU. Physicians (MDs) and fellow MDs were grouped together and NNPs, RNs and RTs were grouped together and a dependent samples t-test was used to examine differences in per cent burnout between these two caregiver groups within each NICU (physicians vs non-physicians).
Objective 3—Examine the relationships between burnout and patient safety culture
Staff burnout may significantly influence the culture of safety in a given work unit. We therefore assessed NICU-level associations between the per cent of respondents reporting burnout and the per cent reporting (A) positive teamwork climate, (B) safety climate, (C) job satisfaction, (D) perceptions of hospital management, (E) stress recognition, and (F) working conditions (the six safety culture scales of the SAQ). Associations were examined using Pearson's zero-order correlations.
Statistical analyses were performed using SAS (V.9.3; SAS Institute, Cary, North Carolina, USA), IBM SPSS Statistics (V.20; IBM, Armonk, New York, USA), and MPlus (V.5.21; Muthen & Muthen, Los Angeles, California, USA). The study was approved by the Institutional Review Boards at Stanford University and Baylor College of Medicine.
Results
Objective 1—Test the psychometric properties of the burnout scale in the NICU setting
There were 2073 surveys returned from the 44 participating NICUs. Of the 44 NICUs, 10 (22.7%) were designated as regional NICUs, 28 (63.6%) as community NICUs and 6 (13.6%) as intermediate NICUs as defined by the California Department of Healthcare Services. These designations are roughly equivalent with designations by the American Academy of Pediatrics as level 4, 3 and 2 respectively.33 Overall response rate is 62.9% (2073 out of 3294), with a range across the 44 hospitals of 21.7% to 100% (mean=69.7%, SD=19.8%). Table 1 exhibits a breakdown of demographics at the NICU and respondent levels.
The internal reliability of the four-item burnout scale in the NICU setting was good with an α=0.85 for the overall sample. The role specific αs range from 0.66 to 0.87 (MD α=0.81, fellow MD α=0.66, NNP α=0.87, RN α=0.85, RCP α=0.86).
Results of the multilevel confirmatory factor analysis collectively suggested that a one factor solution at the within-NICU and between-NICU levels provides an adequate fit to the data, χ2(12)=2622.3, p<0.0001, CFI=0.96, Tucker-Lewis index (TLI)=0.89, RMSEA=0.11, SRMRwithin=0.03 and SRMRbetween=0.03. Standardised factor loadings of burnout items demonstrated a single factor structure, ranging from 0.65 to 0.89 within NICUs and from 0.90 to 1.00 between NICUs.
Results for the intraclass correlation coefficients were: ICC-1=0.13 (95% CI 0.09 to 0.19) and ICC-2=0.95 (95% CI 0.93 to 0.97). The sizeable ICC-2 suggests the appropriateness of aggregating our data to the NICU level and interpretation of this scale as a ‘burnout climate’ scale similar to safety climate or teamwork climate.16 ,34
Objective 2—Provide burnout and resilience benchmarking data for NICUs and caregiver types
Table 2 shows the responses to the Emotional Exhaustion scale of the Maslach Burnout Inventory at the individual and NICU levels. Overall, 27.8% of respondents reported burnout, with a range between NICUs of 7.5% to 54.4% (mean = 25.9%, SD = 10.8). As shown in figure 1, burnout varied significantly between NICUs (F (43, 2029) = 2.86, p<0.0001). Of the staff 49.9% reported being resilient. Resilience also varied significantly between NICUs, F (43, 2029)=2.95, p<0.0001.
A significantly lower percentage of physicians report burnout (mean=15.1%, SD=19.6), relative to RNs, NNPs and RNs (mean=26.9%, SD=11.4, p=0.0004). Physicians also rated themselves significantly more resilient (mean=66.9%, SD=24.8) relative to RNs, NNPs and RNs (mean=51.3%, SD=13.8, p=0.0005).
Objective 3—Examine the relationships between NICU caregiver burnout and patient safety culture
Table 3 summarises the relationships between burnout and safety culture at the NICU level. We found significant associations with (A) teamwork climate, (B) safety climate, (C) job satisfaction, (D) perceptions of hospital management, and (E) working conditions. Specifically, NICUs with a greater percentage of respondents reporting burnout had a smaller percentage of respondents reporting positive teamwork climate (r=−0.48, p=0.001), safety climate (r=−0.38, p=0.01), job satisfaction (r=−0.64, p<0.0001), perceptions of management (r=−0.50, p=0.0006) and working conditions (r=−0.45, p=0.002). Burnout was not related to stress recognition (r=0.12, p=0.44).
Discussion
Our study shows a high prevalence of burnout among NICU personnel, especially among nurses, nurse practitioners and respiratory care providers. Additionally, we have demonstrated a significant association between high burnout scores and poor culture of safety scores. We also establish the reliability and construct validity of a convenient and parsimonious four-item burnout scale. The correlations between burnout and five of the six safety culture dimensions studied help to understand its role in generating a culture of safety. Perhaps not surprisingly, the negative relationship was strongest between burnout and job satisfaction, but the moderate links to teamwork, trust in leadership and working conditions were remarkable. We believe that safety culture is important as an indicator and predictor of care quality, and burnout may also play a role in providing high quality, safe care.
The Maslach Burnout Inventory has 22 items, divided in three scales: emotional exhaustion, depersonalisation and personal accomplishment. In order to reduce survey fatigue among staff, it is important to minimise respondent burden. Of the three scales, emotional exhaustion has been most strongly associated with clinical outcomes.35 The four-item version of the Emotional Exhaustion scale was reliable and appropriate for aggregation to the unit level across the psychometric tests we conducted. In fact, the level of congruence suggests that this four-item version could be used as a metric of ‘burnout climate’ within a unit. Based on the ICC-2, burnout climate behaves similarly to safety climate and teamwork climate, in that it represents not just an individual level construct but a group level construct.16 ,34 Burnout climate, conceptualised this way, can be used for benchmarking.17 Anecdotally, we have found it useful to conceptualise burnout climate as a way to assess group level readiness for change. In our experience, the success of quality improvement work often suffers in clinical areas where a significant proportion of caregivers are burned out. In such situations it may be useful to focus on interventions to ameliorate burnout before or in conjunction with clinical improvement work.
Our finding that 26% of NICU personnel are burned out is within the range of the extant literature. Using a different tool, the Link Burnout Questionnaire, a recent study from Italy among 110 neonatologists found that 30% of respondents experienced high levels of burnout and 60–65% were at risk for burnout.8 Aiken and colleagues, using the Maslach Burnout Inventory, assessed emotional exhaustion among more than 10 000 nurses of surgical patients in Pennsylvania and found that 43.2% had levels above the published ‘high’ norm for medical workers. Compared with a probability-based sample of 3442 working US adults, physicians were more likely to have symptoms of burnout (37.9% vs 27.8%).14
Caregiver burnout has been recognised as an important factor in the well-being and outcomes of healthcare workers and patients (see box 1). This study suggests plausible pathways through which burnout may directly or indirectly affect patient outcomes. The correlations between burnout and safety culture also provide convergent validity in support of the four-item abbreviated burnout scale. Convergent validity measures the degree to which two constructs that should be related are related. Convergent validity supports construct validity, which implies that the inferences made using a measurement tool actually measure the construct being examined.36 All domains, except for stress recognition, were negatively and significantly correlated with burnout, potentially revealing mechanisms through which caregiver lack of well-being may translate into safety lapses, quality deficits and adverse patient outcomes.13 ,37 ,38 The precise mechanisms of these interactions require further exploration and prospective study. However, the ‘climate-like’ nature of burnout suggests that in a clinical area where healthcare workers are resilient, the care context for delivering safe and high quality care may be more favourable. When healthcare workers are less fatigued, more emotionally engaged, and have a better physical sense of well-being, they may be more alert to potential safety hazards and more mindful of their patients’ and families’ needs. They may have better interactions with other staff members, engage more often in unit-wide quality improvement efforts, and be less inclined to leave, resulting in greater continuity of care and better knowledge of their patients. Ultimately, a resilient workforce may strengthen patient safety and quality of care. This study provides a useful metric to assess the impact of novel interventions aimed at improving caregiver burnout or resilience.
Box 1 Impact of burnout in healthcare
Healthcare workers
Higher mortality rates41
More arteriosclerotic disease42
Worse work-life balance43–45
More depression46
More post-traumatic stress11
Worse sleep quality47
Lower teamwork16
Lower job satisfaction12 ,48 ,49
Lower sense of control at work49 ,50
Patients
Higher rates of medical errors13 ,37 ,38
Higher rates of suboptimal care7 ,12
More medical law suits43
Less family support7
Our findings have to be viewed in the light of the study design. First, we included data from only volunteer NICUs, which may bias the results in an unpredictable direction. Second, our sample included only regional, community and intermediate NICUs from California and thus these results may not be generalisable to other NICU types or locations. Third, cross-sectional surveys allow observations and associations to be made, whereas causal relationships between burnout and safety culture cannot be established. Fourth, given the hypothesis generating nature of our study, we didn't have the granular organisational, interpersonal and intrapersonal detail to conduct meaningful multivariate analyses of these relations. This should be done prospectively, in conjunction with hypothesis testing. As the 44 NICUs were a subset of volunteers, the lower level of burnout we detected may be due to selection bias of units where burnout was lower, but this cannot be determined from our currently available results. Finally, the science of assessing and improving safety culture is not yet mature, and our use of the SAQ is based on previous experience.16–18 ,26 ,39 While it is possible that our findings are influenced by non-responder bias, our response rate compares favourably with similar studies.13 ,40 Overall, our findings may inform future studies of burnout among NICU staff, its correlation to clinical outcomes and patient safety events, as well as the testing of interventions to reduce burnout using a more parsimonious scale.
Using a four-item burnout scale, we found a significant prevalence of burnout among NICU caregivers in a sample of 44 California NICUs. Burnout correlated negatively with safety culture, providing a potential gateway into explorations of the link between caregiver well-being and patient safety events and clinical outcomes. The potential utility of burnout climate in future benchmarking may allow users to investigate important trends over time and the effectiveness of targeted interventions.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Contributors JP and JBS had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Individual author contributions: ICMJE criteria for authorship read and met: JP: Acquired funding for this study, conceptualised, designed and executed the study, selected data for inclusion in the survey and analyses, assisted with interpretation of the results, and drafted the initial manuscript. PJS: Helped conceptualise and design the study, was the local lead for California Perinatal Quality Care Collaborative (CPQCC) NICUs, helped select data for inclusion in the survey and analyses, assisted with interpretation of the results, and revised the manuscript. ABA: Analyzed the data, assisted with interpretation of the results and revised the manuscript. MAK: Helped acquire funding for this study, coordinated data collection and study execution, assisted with interpretation of the results, and revised the manuscript. CCN: Coordinated data collection among CPQCC member NICUs, de-identified data, assisted with interpretation of the results and revised the manuscript. EJT: Helped conceptualise and design the study, helped select data for inclusion in the survey and analyses, assisted with interpretation of the results, and revised the manuscript. WC: Conducted the literature review on burnout, assisted with interpretation of the results and revised the manuscript. JBS: Helped acquire funding for this study, conceptualised and designed the study, selected data for inclusion in the survey and analyses, assisted with interpretation of the results, and revised the initial manuscript. All authors approved the final manuscript as submitted.
-
Funding This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (K23 HD056298–01, PI: Profit and K24 HD053771-01, PI: Thomas), Texas Children's Hospital (Paediatrics Pilot Research Fund 33–126, PI: Profit), the Department of Veterans Affairs. (Veterans Administration Center Grant VA HSR&D CoE HFP90-20, Dr Amspoker and Dr Kowalkowski), and the Agency for Healthcare Research and Quality (AHRQ) (1UC1HS014246, PI: Sexton).
-
Competing interests All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf and declare: JP reports grants from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) and Texas Children's Hospital, EJT reports grants from Memorial Hermann Healthcare System and the NICHD, and ABA reports grants from Baylor College of Medicine during the conduct of the study for the submitted work; JP reports personal fees unrelated to the current study from the Vermont Oxford Network, National Quality Forum, and the Christiana Care Health System, and EJT report grants from the Agency for Healthcare Research and Quality outside the submitted work in the previous 3 years; there are no other relationships or activities that could appear to have influenced the submitted work.
-
Ethics approval Stanford University and Baylor University.
-
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Editorial
- Original research
- Original research