Article Text

Neonatal intensive care unit safety culture varies widely
  1. Jochen Profit1,2,3,
  2. Jason Etchegaray4,
  3. Laura A Petersen2,3,
  4. J Bryan Sexton5,
  5. Sylvia J Hysong2,3,
  6. Minghua Mei2,3,
  7. Eric J Thomas4
  1. 1Department of Pediatrics, Baylor College of Medicine, Texas Children's Hospital, Houston, Texas, USA
  2. 2Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
  3. 3Houston Veterans Affairs (VA) Health Services Research and Development Center of Excellence, Health Policy and Quality Program, Michael E DeBakey VA Medical Center, Houston, Texas, USA
  4. 4University of Texas – Memorial Hermann Center for Healthcare Quality and Safety, University of Texas Medical School, Houston, Texas, USA
  5. 5Department of Psychiatry, Duke University School of Medicine, Duke University Health System, Durham, North Carolina, USA
  1. Correspondence to Jochen Profit, Houston Center for Quality of Care and Utilization Studies, VA HSR&D (152), 2002 Holcombe Boulevard, Houston, TX 77030, USA; profit{at}bcm.edu

Abstract

Background Variation in healthcare delivery and outcomes in neonatal intensive care units (NICUs) may be partly explained by differences in safety culture.

Objective To describe NICU care giver assessments of safety culture, explore variability within and between NICUs on safety culture domains, and test for association with care giver characteristics.

Methods NICU care givers in 12 hospitals were surveyed using the Safety Attitudes Questionnaire (SAQ), which has six scales: teamwork climate, safety climate, job satisfaction, stress recognition, perception of management and working conditions. Scale means, SDs and percent positives (percent agreement) were calculated for each NICU.

Results There was substantial variation in safety culture domains among NICUs. Composite mean score across the six domains ranged from 56.3 to 77.8 on a 100-point scale and NICUs in the top four NICUs were significantly different from the bottom four (p<0.001). Across the six domains, respondent assessments varied widely, but were least positive on perceptions of management (3%–80% positive; mean 33.3%) and stress recognition (18%–61% positive; mean 41.3%). Comparisons of SAQ scale scores between NICUs and a previously published adult ICU cohort generally revealed higher scores for NICUs. Composite scores for physicians were 8.2 (p=0.04) and 9.5 (p=0.02) points higher than for nurses and ancillary personnel.

Conclusion There is significant variation and scope for improvement in safety culture among these NICUs. The NICU variation was similar to variation in adult ICUs, but NICU scores were generally higher. Future studies should validate whether safety culture measured with the SAQ correlates with clinical and operational outcomes in NICUs.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

In neonatology there is persistent variation in healthcare delivery and outcomes.1,,7 Differences among neonatal intensive care units (NICUs) with regard to safety culture may in part explain this phenomenon. Preterm infants are fragile, often very ill, and exposed to complex and prolonged intensive healthcare interventions. This makes them vulnerable to lapses in teamwork and patient safety.

The Joint Commission requires that institutions measure their safety culture on an ongoing basis.8 The Safety Attitudes Questionnaire (SAQ) is one of the most widely used instruments to measure safety culture,9 has good psychometric properties10 and is responsive to interventions.11 12 In the adult care setting, a growing literature links improvements in safety culture to improvements in care, such as reduced medication errors, length of stay, nursing turnover rates13 and central line associated blood stream infections.14

What is already known on this topic

  • Patients receiving care in adult and neonatal intensive care units (ICUs) experience wide variation in clinical care and outcomes.

  • In adult ICU settings, higher safety culture ratings have been associated with safer care and better clinical outcomes.

What this study adds

  • In this sample of neonatal intensive care units (NICUs), safety culture varied significantly and revealed widespread opportunities for improvement.

  • NICUs generally had higher safety culture domain scores than adult ICUs.

Unfortunately, investigations of safety culture and teamwork in the NICU are uncommon.15,,18 Safety culture instruments, such as the SAQ, should be applied to specific settings to examine and ensure that their results are meaningful, interpretable, actionable and reliable. This study adds to the empiric body of research by describing variation of safety culture domains in a sample of 12 NICUs, highlights opportunities for improvement, and investigates the influence of respondent characteristics on safety culture scores. Additionally, we investigate the use of an SAQ composite measure, scale scores and comparisons within and outside NICUs.

Methods

Sample, procedure and measures

Please refer to our accompanying paper in this issue for a detailed description of the patient sample, study procedures and instrument measures (data for both papers were handled in the same manner) (in press).19 In brief, the SAQ (ICU Version) was administered to all care givers in 12 NICUs in a faith-based non-profit health system in 2004 using procedures which had previously yielded high response rates.20 Institutional Review Board approval was obtained for this research.19

Analysis

This study addresses the following research questions (RQ) through a secondary analysis of prospectively collected data.

RQ 1: What respondent characteristics are associated with a composite score of safety culture?

For each NICU, a ‘composite score’ was constructed as the arithmetic mean of the scale scores. We explored an aggregation of SAQ scores to a composite score19 following the methods of Vogus et al.21

We investigated associations between the composite score and respondent characteristics in bivariate analyses and input characteristics associated at p≤0.1 in a hierarchical model. Respondents were nested within NICUs. The multivariate model was developed to adjust for potential confounding of scale scores due to respondent characteristics. A two-tailed p value <0.05 was considered statistically significant.

RQ 2: To what extent do NICUs in the same hospital system share common assessments of patient safety and teamwork related norms?

Mean scale scores were computed for each NICU. We tested for statistical differences in scale scores between NICUs ranking among the top and bottom four NICUs. Unadjusted analyses compared the combined means of relevant NICUs via Student t tests. Adjusted analyses used fixed effect linear models informed by the previous multivariate analysis to compare adjusted mean scale scores between the top and bottom four NICUs.

RQ 3: Which domains and items of the SAQ signal the greatest opportunities for improvement?

For comparability with previously published literature using the SAQ, we present scale and item level percent positives (percent of respondents answering ‘agree slightly’ or ‘agree strongly’). SAQ scale scores 60% positive or higher are considered favourable, with a goal of at least 80% positive. In addition to the 12 NICUs reported here, we also evaluated the percent positive scale scores between the NICUs and a previously published cohort of 71 adult intensive care units (ICUs)22 using independent samples t tests.

Results

RQ 1: What respondent characteristics are associated with a composite score of safety culture?

We received completed surveys from 547 of 639 respondents; the overall response rate was 86% (range 69%–100%). Table 1 presents the response rates and respondent characteristics. Online supplementary appendix 1 displays respondent characteristics by site. After adjusting for respondent characteristics, only NICU site was independently associated with the composite. There was a trend that job position was associated with the composite (p=0.06). Comparisons between job positions and the composite revealed that, on average, nurses and ancillary personnel rated safety culture 8.2 (p=0.04) and 9.5 (p=0.02) points lower than doctors. There was little difference between nurses and ancillary personnel. Figure 1 illustrates the differences in composite safety culture scores between nurses and doctors by NICU.

Figure 1

Safety Attitudes Questionnaire composite score by neonatal intensive care unit (NICU) and job position. Hierarchical regression model (respondents nested within NICUs). The composite score is the average of the mean scale scores.

Table 1

Respondent characteristics and associations with the safety culture composite score

RQ 2: To what extent do NICUs in the same hospital system share common assessments of patient safety and teamwork related norms?

We found wide variation in mean scores across each of the domains, and even more so between percent positive scores (see table 2). Safety culture composite scores varied by over 20 points across NICUs and individual scale scores ranged even greater. After adjusting for job position, patient safety attitudes varied widely between the top and bottom four NICUs across all domains. For illustration, we show mean item scores from the teamwork and perception of management scales (see table 3).

Table 2

Composite and mean scale scores

Table 3

SAQ item descriptives used for perceptions of management and teamwork climate*

RQ 3: Which domains and items of the SAQ signal the greatest opportunities for improvement?

Respondent and NICU level variability by scale and item are shown in table 4. These items are used for the calculation of scale scores and constitute the SAQ Short Form, the version most widely used at this time. On average, about half of the respondents in a given NICU reported a good teamwork climate (54%) and a good safety climate (55%), one out of three reported positive perceptions of management (33%), and almost two out of three reported good job satisfaction (63%).

Table 4

Distribution of responses by NICU and across all respondents

The lowest percent positive scale score across the 12 NICUs was for perceptions of management: 37% agreed that administration supported their daily efforts, and 43% agreed that staffing levels were sufficient to handle the number of patients.

Compared with the adult ICU sample, NICUs exhibited significantly higher percent positive (percent agreement) domain scores for safety climate (t=−2.588, p=0.011), job satisfaction (t=−2.043, p=0.044) and working conditions (t=−2.298, p=0.024) (figure 2).

Figure 2

Safety Attitudes Questionnaire percent positive scale scores by neonatal intensive care unit (NICU) and state-wide sample of adult intensive care units (ICUs) from Michigan (adapted from Sexton et al22). NICUs are marked with hashes.

Scale: ICU mean percent positive SD; range), NICU mean percent positive (SD; range).

Discussion

Key findings reveal: (1) significant variation in safety culture domains among this sample of NICUs; (2) opportunities for improvement within all domains measured by the SAQ; (3) patterns of more positive safety culture domains in NICUs relative to a large sample of adult ICUs from the same time period; and (4) a trend for an association between job position and safety culture where physicians assess teamwork and safety related norms more positively than nurses and other ancillary personnel.

Variation and implications for improvement

Our finding of significant variation in the patient safety culture among NICUs is consistent with the existing literature. We previously demonstrated significant between and within hospital variation in safety culture domains among 203 clinical sites from healthcare facilities in the UK, the USA and New Zealand,10 consistent with the work of Singer and colleagues.23 24

Based on the mean scale scores, the safety profile of the participating NICUs was quite variable in that some NICUs excelled in one subscale but lagged in another. For example, NICU E may find opportunities for improvement in interventions that address teamwork climate, whereas NICU F may focus on stress recognition and perceptions of management. This implies that one safety culture intervention bundle is unlikely to be useful for all; rather, interventions may need to be tailored to individual NICU profiles. Moreover, a composite index of the six safety culture domains, while statistically justifiable, may significantly dilute the domain level variability within an NICU.

The variation demonstrated across the 12 NICUs suggests opportunities for improvement in safety culture and quality of care. A growing literature demonstrates such a link. For example, the SAQ has demonstrated sensitivity to quality improvement interventions in the operating room25 and obstetric setting,12 and has been associated with reduced blood stream infections,14 medication errors and lengths of stay.13 In addition, safety culture has been shown to predict success in implementing complex quality improvement projects,22 implying that attention to safety culture may be a necessary co-intervention for many quality improvement projects.

Specific opportunities for improvement

Responses to the SAQ exhibited two general themes: perceived lack of support from hospital management and concerns about aspects of teamwork and collaborative communication regarding errors. These issues are amenable to intervention that could be studied in future work.

With regard to perceptions of management, many providers indicated a lack of support in their daily efforts by management and mentioned that staffing may be insufficient to handle the work load. Managers may not easily be able to remedy such grievances because of financial constraints. However, administrators could employ several managerial tools to motivate staff and create a supportive work environment in which clinical providers feel that their concerns are being considered.26 Management is essential to promoting an environment of collegiality, teamwork and common purpose. Good teamwork climate scores have been associated with lower levels of clinical care giver burnout from their work, which may have important implications for nurse retention and innovation readiness.27

Additionally, item level scores from the teamwork and perception of management domains have been shown to be sensitive to interventions. For example, collaborative rounds,28 aviation based crew resource management training29 or improved communication across hierarchies30 31 can improve teamwork, whereas Leadership WalkRounds32 33 can improve perceptions of management.

Comparisons between NICUs and adult ICUs

Compared to adult ICUs, NICU percent positive scale scores were higher regarding safety climate, job satisfaction and working conditions. NICU teamwork climate and perceptions of management trended toward higher scores, although they did not reach statistical significance. Stress recognition scores were relatively low in NICUs and ICUs, and did not significantly differ between them. Overall, NICU and adult ICU samples exhibited similar amounts of variability, and the lowest percent positive scores for NICUs (perceptions of management and working conditions) were also the lowest percent positive scores for adult ICUs.

Differences in safety attitudes by job position

Consistent with most reports across a variety of healthcare settings, NICU physicians rated patient safety culture significantly higher than nurses and ancillary personnel.10 16 24 34 35 In previous work, we found that compared to physicians, nurses were less comfortable with speaking up, thought their input was not well received, felt conflict resolution was often inappropriate, and desired more input to decision-making.35 Singer and colleagues suggested that nurses may experience safety deficiencies more acutely because of their tighter employment relationship with the hospital.24

Since hospitals are required to assess their safety culture on an ongoing basis, safety culture surveys might be used for comparing hospitals or units within hospitals against their peers. Systematic differences in responses by job position suggest that NICU comparisons should use consistent survey inclusion criteria such that representative and meaningful results can be garnered through questionnaire assessments. This would reduce the ability of an NICU to improve its ratings artificially by preferentially asking physicians to respond.

The results of this study must be seen within the context of its observational design. Associations between safety attitudes and other variables do not necessarily indicate causality. Our findings may be confounded by unobserved variables, such as respondent income or experience in other healthcare settings. Within the confines of the available dataset, we have tried to maximise internal validity by adjusting for possible confounding using multivariate modelling. Our accompanying paper further supports the use of SAQ scale scores as statistically valid consensus views for a specific NICU.19

Administration of the SAQ during departmental meetings may potentially introduce unwanted selection and response bias due to the social desirability of positive responses and the sample of providers who attend the meetings. However, if such bias existed, it would have likely resulted in higher SAQ scores, therefore strengthening our overall conclusions.

Finally, we studied a relatively small and non-random sample of care providers in 12 NICUs. It is unclear whether our results are generalisable to other NICUs, but the significant variability found here within a small sample of NICUs improves generalisability.

Conclusions

We found significant variation and scope for improvement in safety culture domains among this sample of NICUs. The NICU variation was similar to the variation in adult ICUs, but NICU scores were generally higher than adult ICU scores. Future studies should validate whether safety culture correlates with clinical and operational outcomes in the NICU setting.36

Acknowledgments

In addition to thanking the NICU personnel who participated by sharing their assessments, the authors would like to acknowledge the contribution of the study staff, Christen Fullwood, Chris Holzmueller, Angelina Barbosa and Linda Marcellino.

References

View Abstract

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

    • Web Only Data - This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Funding JP's contribution is supported in part by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (#1 K23 HD056298-01, PI: Profit). LAP was a recipient of the American Heart Association Established Investigator Award (#0540043N) at the time this work was conducted. LAP, SJH and MM also receive support from a Veterans Administration Center Grant (VA HSR&D CoE HFP90-20). SJH's contribution is supported in part by the Department of Veterans Affairs Health Services Research and Development Program (#CD 2-07-0818). EJT's effort is supported in part by grants from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (#1 K24 HD053771-01, PI: Thomas and #1 PO1 HS1154401, PI: Thomas). JBS received support from an Agency for Healthcare Research and Quality (AHRQ) (grant # 1UC1HS014246). JE's effort is supported by a K02 award from AHRQ (#1 K02 HS017145-02) and the University of Texas at Houston – Memorial Hermann Center for Quality and Safety.

  • Competing interests None.

  • Ethics approval The original study was approved by the Johns Hopkins University Institutional Review Board, and the analysis of a de-identified data set was approved by the Institutional Review Board at Baylor College of Medicine.

  • Provenance and peer review Not commissioned; externally peer reviewed.