Purpose To determine the effectiveness of patient safety culture strategies to improve hospital patient safety climate.
Data sources Electronic search of the Cochrane Library, OVID Medline, Embase, CINAHL, proQuest and psychinfo databases, with manual searches of quality and safety websites, bibliographies of included articles and key journals.
Study selection English language studies published between January 1996 and April 2011 that measured the effectiveness of patient safety culture strategies using a quantitative measure of patient safety climate in a hospital setting. Studies included were randomised controlled trials (RCTs), non-RCTs, controlled before and after studies, interrupted time series and historically controlled studies.
Data extraction Data extraction and critical appraisal were conducted by two independent reviewers. Study design, intervention, level of application, setting, study participants, safety climate outcome measures and implementation lessons were extracted from each article.
Results of data synthesis Over 2000 articles were screened, with 21 studies meeting the inclusion criteria, one cluster RCT, seven controlled before and after studies, and 13 historically controlled studies. There was marked methodological heterogeneity amongst studies. Impacts of 11 different strategies were reported. There was some evidence to support that leadership walk rounds (p=0.02) and multi-faceted unit-based programmes (p < 0.05) may have a positive impact on patient safety climate.
Conclusions Despite strong face validity for a variety of patient safety culture strategies, there is limited evidence to support definitive impacts on patient safety climate outcomes. Organisations are advised to consider robust evaluation designs when implementing these potentially resource intensive strategies.
Statistics from Altmetric.com
There is a current focus on measuring and improving patient safety culture to enhance patient safety in hospitals.1 This is reflected in the increasing number of literature reports on patient safety culture performance.2–13 Patient safety culture is encouraged at jurisdictional and organisational levels by national health policy makers, with hospitals routinely administering surveys in many countries.14–16 Patient safety culture (figure 1), a component of organisational culture, includes the shared beliefs, attitudes, values, norms and behavioural characteristics of employees17 and influences staff member attitudes and behaviours in relation to their organisation's ongoing patient safety performance.18–21
Accurate measurement of patient safety culture is limited by the ability to define measureable components of culture.22 Therefore the demand for relatively low-cost, quick and easy to use assessments of patient safety culture has resulted in a reliance on patient safety climate questionnaires.6 ,23–28 Patient safety climate (figure 1) describes employee perceptions and attitudes about the surface features of patient safety culture at a given point in time.29 A number of patient safety climate questionnaires have been developed29 ,30 and used within healthcare organisations to measure performance for benchmarking, diagnosis and planning of internal quality improvement. More recently they have been used to examine the effectiveness of strategies designed to improve patient safety culture.
Positive patient safety climate has been reported to be associated with enhanced patient safety.31–34 Therefore, targeting practice change through patient safety climate is considered to be a key strategy for strengthening and enhancing patient safety and outcomes in hospitals.1 ,14 ,35 However, it is important that enthusiasm to introduce strategies for improving patient safety culture and climate is informed by evidence of effectiveness. Despite the application of a variety of patient safety culture strategies within hospitals there has been no prior systematic review of their effectiveness. As some strategies may entail significant resource commitment to implement, embed and sustain, it is important to determine the extent to which there is evidence to support their effectiveness, generalisability and sustainability to enable rational allocation of resources.36
The aim of this systematic review was to critically assess the evidence for the effectiveness of patient safety culture strategies for improving patient safety climate in hospitals, to support decision-making by organisations and funding providers.
An electronic search was conducted of the Cochrane Library, OVID Medline, Embase, CINAHL, proQuest and psychinfo databases. Medical subject heading search terms and keywords were used (online appendix 1). The Medline search strategy was translated for other databases as appropriate. Additionally, bibliographies of included articles and key journals were hand searched.
English language studies published between January 1996 and April 2011 were considered. Assessment of effectiveness was based on randomised controlled trials (RCTs), non-RCTs, controlled before and after studies, interrupted time series and historically controlled studies.
Studies were included if they measured effectiveness of patient safety culture strategies using a quantitative patient safety climate tool and were conducted within a hospital, hospital department or clinical unit.
Studies without a quantitative measure of patient safety climate; studies in community, primary healthcare services or residential care facilities; and case studies that presented only cross-sectional data.
Data extraction and analysis
Data extraction and critical appraisal of included studies were conducted by two independent reviewers (RM and JL), with disagreements settled by a third reviewer (AB). Assessment of risk of bias and study critical appraisal was conducted using a tool based on National Health and Medical Research Council (NHMRC)37 and the Cochrane Collaboration's Effective Practice and Organisation of Care (EPOC) Group guidelines.38 ,39 Study setting, design, selection and measurement bias, baseline outcome measurements and characteristics, risk of contamination, data analysis, selective outcome reporting, other risks of bias and issues relating to generalisability and sustainability were extracted and recorded. Implementation lessons and data from studies using mixed method evaluations were also extracted and reviewed. A meta-analysis was not possible due to insufficient homogeneity (populations, interventions, outcome measures and follow-up periods) of studies. Consequently, data were reviewed looking for common themes and presented in a narrative format.
Over 2000 articles were screened for inclusion, of which 21 studies (22 publications), one cluster RCT, seven controlled before and after studies and 13 historically controlled studies met the inclusion criteria (figure 2). Included studies (see online supplementary table S1) were conducted in the USA (15 studies),34 ,40–53 UK (3 studies),54–57 Canada (1 study),58 Europe (1 study)59 and Australia (1 study).60
Study periods ranged from 2 months to 3 years. Seven of the 21 studies had observation periods longer than 12 months.43 ,47 ,49 ,51 ,53 ,54 ,59 Surveys used (or adaptations of) included the Safety Attitudes Questionnaire (SAQ),40 ,42 ,44 ,47 ,49–53 ,55 ,56 the Safety Climate Survey (SCSu),45 Patient Safety Cultures in Healthcare Organizations,43 ,58 the Hospital Survey on Patient Safety Culture,41 ,46 ,59 ,60 and the National Health Service National Staff Survey.54 ,57 All tools had undergone varying levels of validation.29 ,30 Four studies conducted factor analysis to assess internal consistency and content reliability of items included on tools used.34 ,43 ,58 ,59
There were a number of different safety culture strategies tested, including leadership walk rounds,42 ,47 ,55 structured educational programmes,55 ,58 team-based strategies,41 ,44 ,53 simulation-based training programmes,43 ,45 ,48 ,56 multi-faceted unit-based programmes40 ,46 ,49–53 and multi-component organisational interventions54 (see online supplementary table S1). Six studies specifically focused on teamwork within the clinical area,41 ,44 ,55 ,56 ,60 while others focused on varying factors of patient safety culture, such as incident reporting, environment, human factors and leadership. One two-phase study involved an intervention implemented at the organisational level,54 ,57 while the remaining 20 studies implemented interventions at the unit or department level. Four of the eight studies that incorporated a contemporary control group reported limited or no effects on measures of patient safety climate.42 ,44 ,55 ,58
Summary of study findings
Leadership walk rounds
Thomas et al's cluster RCT examined the effectiveness of leadership walk rounds,42 finding no effect on patient safety climate scores reported by doctors, nurses and clinicians. However, a positive effect on patient safety climate was detected for nurses participating in leadership walk rounds compared with those in the control group (p=0.02).
Frankel et al's historically controlled study supported these findings,47 reporting an increase in mean safety climate scores following introduction of leadership walk rounds.
Structured educational programmes
Bleakley et al's controlled before and after study examined the effectiveness of a structured educational programme on teamwork collaboration delivered to operating theatre staff.55 There was a positive effect on teamwork climate (one of six dimensions of the SAQ) (p=0.04), however there were significant differences in baseline measures between groups. Ginsburg et al's controlled before and after study examined the effectiveness of a nurse clinical leader education programme,58 finding an effect on one of three dimensions of the patient safety climate survey (valuing safety: p<0.001).
O'Leary et al's controlled before and after study examined the effectiveness of daily structured interdisciplinary rounds, reporting no effect on patient safety climate scores.44 However, a positive effect was found for the SAQ team work climate dimension (p=0.01).
Weaver et al's controlled before and after study examined the effectiveness of TeamSTEPPS (team strategies and tools to enhance performance and patient safety),41 finding a positive trend to improvement of patient safety climate scores over time for both intervention and control groups (p < 0.001), with no significant difference between the two groups.41
Stead et al's historically controlled study reported a positive effect on 2 of 12 dimensions of patient safety culture (frequency of event reporting, p=0.04 and organisational learning, p=0.01).60
Simulation-based training programmes
Cooper et al's controlled before and after study examined the effectiveness of a simulation-based training programme, and reported no effect on patient safety climate scores.43 Four historically controlled studies reported varying levels of effectiveness of simulation-based training on patient safety climate.45 ,48 ,53 ,56
Multi-faceted unit-based programmes
Pronovost et al's controlled before and after study examined the effectiveness of a structured multi-faceted unit-based safety programme (structured framework for assessing, identifying, reporting and improving patient safety concerns),40 finding a positive effect on safety climate scores (p<0.05).
Six historically controlled studies supported these positive findings of effectiveness for multi-faceted unit-based programmes.46 ,49–53 All studies reported varying levels of improvements in at least one dimension of patient safety climate over time.
Multi-component organisational interventions
Benning et al's two-phase controlled before and after study examined the effectiveness of a multi-component organisational intervention, the Safer Patient Initiative, involving organisational wide and targeted components in designated clinical areas, such as approaches to build safety culture and good leadership, reducing medication errors, and enhancing communication to reduce adverse events.61 A negligible effect on patient safety climate scores was reported.54 ,57
Other patient safety culture strategies
Two historically controlled studies reported positive impacts with other patient safety culture strategies, surgical safety checklists and improvement approach strategies (see online supplementary table S1).34 ,59
Online supplementary table S2 summarises the evidence for the effectiveness of patient safety culture strategies.
Qualitative data and implementation lessons
Six studies contributed qualitative data about implementation issues to support quantitative findings of effectiveness of patient safety culture interventions (see online supplementary table S1),40 ,41 ,52 ,57 ,58 ,60 with only three studies using formal mixed methods evaluation.41 ,54 ,60 Benning et al's controlled before and after study conducted a series of focus groups, interviews and observations with clinical ward staff.57 These data highlighted gaps between management level engagement, ward practice and clinical staff engagement and practice change, in relation to effective implementation of a multi-component organisational intervention across all hospitals in phase I of the trial. The authors stated that, in general, it was thought hospitals had underestimated the resource and organisational support required to make the patient safety initiatives work and achieve culture and practice change.
Stead et al and Weaver et al conducted observational analyses on team behaviours following the implementation of TeamSTEPPS.41 ,60 These studies reported improvements in team behaviours and the structure and process of team meetings post implementation.
The evaluation of the studies has been presented in a summary table (online appendix 2), which includes information regarding study characteristics, a summary of key findings, the effect of the intervention and assessment of the quality of evidence.
All studies had a number of methodological limitations. These included non-equivalent contemporary control groups or use of a historical control group, risk of selection bias, small sample sizes, limited follow-up response rates, short observational periods, and use of post hoc statistical analysis for examination of effect. Most studies failed to adequately control for potential threats of bias or confounding factors, which threatens their internal validity. The factors identified were sufficient to impact on the interpretation of study results. Furthermore, 9 of the 21 studies were conducted in single hospital or clinical units, limiting generalisability of results.
This systematic review found limited evidence to support the effectiveness of a variety of in-hospital patient safety culture strategies, the impact of which was assessed using patient safety climate scores. A variety of strategies were tested that targeted a diverse range of issues, including leadership, accountability, communication and teamwork, identification of safety concerns, reporting of near misses and reliability. This array may reflect the complexity of the construct we intuitively understand as ‘culture’ and the current lack of understanding of the priority of certain elements and their potential interdependencies.
Leadership walk rounds and multi-faceted unit-based strategies were the two strategies for which some stronger evidence could be found to support a positive impact. Leadership walk rounds engage executives and clinical leaders and use direct communication to break down barriers surrounding patient safety, whilst engaging frontline care givers and demonstrating organisational commitment.62 Thomas et al concluded that the positive impact on patient safety climate was limited to the nursing staff who actually participated in the leadership walk rounds.42 The limited effectiveness and the relatively short observation period of 3 months may be reflective of the time it takes for culture changes to become embedded within an organisation. However, it may also raise implementation issues for organisations with large numbers of agency staff or high staff turnover, when some staff may be rarely exposed to a walk round. Frequency of rounding may also be relevant as supported by data from a historically controlled study47 in which weekly walk rounds were undertaken for 18 months compared with the Thomas study in which they were conducted monthly for 3 months.
There is also some evidence to support the use of multi-faceted unit-based programmes for improving patient safety climate and patient outcomes. These programmes apply a structured framework to assess, identify, report and improve patient safety defects and the strategy is specifically designed to improve unit culture.40 The clear relationship between this strategy and patient safety climate is likely to underpin the observed positive impacts compared with other strategies when the relationship with patient safety climate outcomes is not clear or may be indirect. A number of historically controlled studies have been conducted using both small and large cohorts of clinical units.49–52 While the lack of data from a contemporary control group limits assessment of efficacy, it does support feasibility for implementing and transferring this unit-based safety programme.
Many of the remaining strategies were examined by only a small number of studies with conflicting results. It has been recognised that there is a need for culture strategies to be more selective and flexible, and able to be adapted to contextual factors and the climate in which they are being delivered.19 ,63 Developing a culture within which an organisation can achieve patient safety is challenging5 and the sustainability of many of these strategies over time is unclear. Hospitals are dynamic in nature and inherently hierarchical in structure.1 Culture is often deeply embedded within various levels and sub-groups.8 ,19 ,20 Therefore, to modify culture it is important to first understand the current hierarchy of these elements in the target group for the intervention.17 Commonly reported explanations for limited effectiveness were gaps between what managers reported they were doing and what was actually happening at the local level,42 ,57 and limited implementation of safety initiatives due to competing priorities and resource constraints.51 ,58
It should also be noted that the patient safety culture strategies identified in this review are likely to be highly dependent on the personnel conducting them and the organisation in which they are conducted. Intrinsic organisational elements such as the level of executive support, extent of implementation, the size of the organisation and the personnel facilitating change may limit the generalisability of evidence. Even strategies with a strong evidence base will fail in an organisation when they have been poorly implemented or supported.
Three areas worthy of further discussion are the methodological rigour of the studies, the limited number of studies in this area, and the robustness of the safety climate measures used. Over 60% of identified studies relied on historically controlled study designs, often in single organisations, making it difficult to directly attribute the observed effect on patient safety climate to the patient safety culture strategies implemented.36 Such studies are highly exposed to potential sources of bias and confounders. Underlying factors can produce large fluctuations in the outcomes of interest. As such inferences of causal associations are difficult to determine due to the variety of variables that may exist within an organisation or clinical unit. Of note, some studies found a time effect but not an intervention effect on patient safety climate.41 ,54 This suggests that survey tool administration alone may have a positive impact on safety climate because it may sufficiently increase staff awareness about positive culture and patient safety, enticing cultural change.
While we recognise the difficulties in designing, conducting and financing health service research of complex interventions, such as patient safety culture strategies, a relatively small number of studies were identified addressing their effectiveness. It can be argued that potentially costly and resource-intensive quality improvement programmes should be subject to the same scrutiny as clinical practice.36 In an industry with limited resources, implementation of a non-evidence-based practice should be approached with caution. It is important to note that limited evidence in this area does not equate to lack of effectiveness of patient safety culture strategies. However, while there may appear to be sufficient face validity for these strategies, there is always a danger that strategies may not be as effective as first thought and may also pose high costs.64 As such, further research in this area is required. Ideally, patient safety culture strategies would benefit from evaluation using quantitative and qualitative methodology.22 ,65 ,66 Qualitative data can be used to support quantitative findings and the contextual factors influencing effectiveness. Only 3 of the 21 included studies conducted formal mixed-methods evaluations.57 ,60
Reviews of safety climate tools used within healthcare have indicated a large disparity between the levels of measured psychometric properties, safety climate dimensions and the theoretical underpinning for questionnaires.29 ,30 ,67 There was considerable diversity in the patient safety climate measurement tools used by studies and this limits the degree to which between-study comparisons can be made. Emerging evidence links safety climate score with patient outcomes,31–34 however the extent to which these measures can predict patient safety outcomes remains somewhat unclear. There also appears to be limited understanding of survey tools’ sensitivity and specificity to detect change and the evidence to associate changes in patient safety climate with changes in patient safety outcomes is scarce. Organisations looking to evaluate patient safety culture using patient safety climate tools should consider the degree to which that tool has been validated within their jurisdiction and application of these tools and their appropriateness for evaluating effectiveness should also be considered in future research.
The strengths of this review, which focused on quantitative measures of effectiveness, include a systematic search of 15 years of peer-reviewed literature with a rigorous approach to critical appraisal of study design, bias and contamination, outcome measures, methods of analysis and reporting. However, the limitations of this review must also be considered. First, the exclusion of studies without any quantitative measure of patient safety climate meant that studies solely conducting qualitative research or measures of patient safety, such as mortality rates, medication errors, and patient complaints, were not included. We acknowledge that qualitative research does provide useful insight, particularly into barriers and facilitators for implementation, and identifying the effectiveness of patient safety culture strategies on measures of patient safety is important to consider. However, it was beyond the scope of this review to include these studies. Future systematic reviews should include such measures to augment the findings of this review. Second, while the search strategy appeared comprehensive, other relevant studies may have been inadvertently excluded; however, our manual search yielded only six additional studies, suggesting recall sensitivity and precision were adequate.
Implications for future research
A variety of strategies and initiatives are being implemented in healthcare organisations without clear knowledge of effectiveness. Research efforts should focus on strengthening the evidence around the effectiveness of strategies to change patient safety culture, ideally using quantitative and qualitative methods. Although issues related to patient safety are common across jurisdictional healthcare organisations, a central approach of implementation within a cluster randomised or wedge-stepped design could support appropriate evaluation. At an organisational level, when such designs are not feasible, continued efforts by healthcare organisations to pursue improvements in patient safety culture are recommended, within a structured evaluation using time series analyses and well planned programme evaluation. This will contribute to understanding the impact of changes over time and the sustainability of chosen strategies.
The evidence is limited to support the effectiveness of strategies to improve patient safety culture within hospitals. However, there is a rationale for further investigating executive walk rounds and multi-faceted unit-based programmes. Organisations considering the implementation of potentially costly and resource-intensive strategies should evaluate programmes within a robust study design.
We would like to acknowledge the comments made by the Editor in Chief, BMJ Quality and Safety. We would also like to thank the reviewers of this manuscript for their comments which have been used to assist in the revision of this paper.
Contributors RM: primary reviewer of included studies (data extraction and critical appraisal), drafting and finalisation of manuscript. JL: secondary reviewer of included studies (data extraction and critical appraisal) and revision of manuscript. AB: third reviewer and review of critical appraisal of studies, revision of the manuscript and finalisation of the discussion section of the manuscript. RMc: assisted with data extraction and critical appraisal of included studies and revision of the manuscript. DD: review and revision of the manuscript, in particular the introduction and methods sections. CB: overseeing the drafting and finalisation of the manuscript, with particular focus on the introduction and discussion sections.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data extraction and critical appraisal of included studies are available on request from the corresponding author.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.