Article Text
Abstract
Background Hospital organisational culture affects patient outcomes including mortality rates for patients with acute myocardial infarction; however, little is known about whether and how culture can be positively influenced.
Methods This is a 2-year, mixed-methods interventional study in 10 US hospitals to foster improvements in five domains of organisational culture: (1) learning environment, (2) senior management support, (3) psychological safety, (4) commitment to the organisation and (5) time for improvement. Outcomes were change in culture, uptake of five strategies associated with lower risk-standardised mortality rates (RSMR) and RSMR. Measures included a validated survey at baseline and at 12 and 24 months (n=223; average response rate 88%); in-depth interviews (n=393 interviews with 197 staff); and RSMR data from the Centers for Medicare and Medicaid Services.
Results We observed significant changes (p<0.05) in culture between baseline and 24 months in the full sample, particularly in learning environment (p<0.001) and senior management support (p<0.001). Qualitative data indicated substantial shifts in these domains as well as psychological safety. Six of the 10 hospitals achieved substantial improvements in culture, and four made less progress. The use of evidence-based strategies also increased significantly (per hospital average of 2.4 strategies at baseline to 3.9 strategies at 24 months; p<0.05). The six hospitals that demonstrated substantial shifts in culture also experienced significantly greater reductions in RSMR than the four hospitals that did not shift culture (reduced RSMR by 1.07 percentage points vs 0.23 percentage points; p=0.03) between 2011–2014 and 2012–2015.
Conclusions Investing in strategies to foster an organisational culture that supports high performance may help hospitals in their efforts to improve clinical outcomes.
- leadership
- quality improvement
- qualitative research
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Hospital organisational culture is associated with patient outcomes,1–4 including lower risk-standardised mortality rates (RSMRs) for patients with acute myocardial infarction (AMI).5 6 Organisational culture encompasses the shared beliefs, values and patterns of behaviour that enable hospitals to survive in complex and changing environments.7 Inter-relationships between the environment and a given organisation’s processes and practices are multifaceted, complex and bidirectional8; hence, the influence of culture may be difficult to measure precisely. There remains an ongoing debate regarding whether culture acts as a contextual factor, a moderator of organisational practices or an independent variable.9–11 In this study, we conceptualise organisational culture as a factor that can accelerate learning and improvement, and therefore impact the adoption of evidence-based practices as well as have a potential direct impact on performance. Despite evidence on the link between organisational culture and patient outcomes, little is known about whether and how culture can be positively influenced in order to reduce AMI mortality.
Previous interventional studies have reported improvements in teamwork and coordination,1 12–15 but these interventions have largely focused on subunits within hospitals, including surgical suites,13 15 emergency departments12 14 or intensive care units.1 Some studies have reported success in improving patient safety culture16–20; however, this research has not examined other aspects of culture linked to performance improvement.21 Efforts such as the Robert Wood Johnson Pursuing Perfection programme illuminated key elements of hospital culture change,22 but were not designed to evaluate a standard intervention to influence culture across sites and over time.23 A 2011 Cochrane review24 noted that prior studies were limited by including single or few sites, lack of validated measures of organisational culture25 and limited longitudinal follow-up to assess impact.
We designed a 2-year intervention, Leadership Saves Lives (LSL),26 directed at fostering changes in hospital organisational culture that might contribute to reductions in RSMRs for patients with AMI. Our study addresses limitations of prior research24 through a longitudinal design in a diverse sample of hospitals, the use of robust quantitative and qualitative measures of culture, and the inclusion of an important clinical outcome.
Methods
Study design and sample
We employed a longitudinal, convergent mixed-methods intervention design.27 Experts recommend mixed methods for the evaluation of complex interventions,28 where quantitative data assess impact and qualitative data provide critical insights into both context29 and processes of change.30 The intervention was not designed as a randomised controlled trial, as multifaceted contextual factors could not feasibly be controlled and are in fact the central phenomenon of interest in this study.28 30 31 We also tracked available RSMR data in non-intervention hospitals nationally over the study period. We partnered with the Mayo Clinic Care Network (MCCN), a national group of regional medical systems committed to quality improvement through collaboration, to identify a sample of hospitals seeking to improve performance on RSMR. From the 21 MCCN members (as of January 2014), we identified those meeting eligibility criteria, including (1) at least 200 AMI discharges per year to ensure sufficient experience in caring for patients with AMI, (2) average or below-average national performance on 30-day RSMR between 1 July 2009 and 30 June 2012 as reported by the Centers for Medicare and Medicaid Services (CMS) Hospital Compare in Spring 2014, suggesting opportunity for improvement, and (3) the largest hospital in the system, for hospitals in multihospital systems.
From the list of 18 hospitals that met eligibility criteria, we used random sampling with a purposive component32 to select hospitals that were diverse in geography and teaching status. We randomised the list using a random number generator and, working in sequence from the top, approached the first 10 hospitals to determine receptivity, skipping those that duplicated prior selections in terms of geography or teaching status; one declined due to reorganisation, and one declined as a competing initiative was underway. We replaced these two hospitals with sites similar in geography and teaching status. This sample size was sufficient to observe substantial variation at the hospital level while allowing adequate resources to conduct the intervention.
The intervention
The intervention, previously described in detail,26 was implemented from 25 June 2014 to 24 June 2016. LSL was designed to foster improvements in five domains of hospital organisational culture relevant to hospital performance: (1) learning environment (ie, climate that promotes and rewards enquiry and experimentation),33 (2) psychological safety (ie, shared belief that it is safe to take risks interpersonally and to speak up without punishment),34 (3) senior management support (ie, fostering a shared purpose and vision for change, and empowering line leaders to enact that vision),35 (4) commitment to the organisation (ie, employees’ desire to stay based on their identification with and attachment to the organisation)36 and (5) time for improvement efforts (ie, space for planning, reflection and feedback).34 Each hospital appointed a guiding coalition of approximately 15 staff involved in care of patients with AMI, from multiple departments (eg, cardiology, emergency medicine, pharmacy, quality improvement) and levels of the organisation (eg, senior executives to front-line staff).
Guiding coalition members participated in three intervention components: a series of three annual forums attended by four members of each guiding coalition; a series of four 1-day, on-site workshops with the full coalition for each hospital; and a web-based platform for sharing experiences. The workshop curriculum (online supplementary appendix A) was grounded in a strategic problem-solving approach37 in which coalitions sought to foster improvements in organisational culture as they implemented evidence-based strategies,5 with the goal of identifying and addressing root causes of AMI mortality to reduce RSMR. To build a culture that supports creative problem solving, workshop content included experiential learning sessions focused on promoting role clarity, working across professional and organisational boundaries, working in hierarchy, creating psychological safety, developing accountability for shared goals and engaging in productive conflict.
Supplementary file 1
Measures and data collection
We measured changes in three outcomes: domains of organisational culture, uptake of evidence-based strategies associated with lower RSMR, and RSMR for intervention hospitals and nationally. To quantify changes in organisational culture, we employed a web-based, validated 31-item survey38 (online supplementary appendix B), reflecting five subscales of domains of culture aggregated into an overall culture score. We surveyed guiding coalition members at baseline (168 individuals), 12-month (186 individuals) and 24-month (178 individuals) waves. Uptake of five evidence-based strategies associated with reducing RSMR for AMI (eg, creative problem solving, monthly meetings with EMS to review AMI cases, pharmacists rounding on all patients with AMI, physician and nurse champions for AMI care, and nurses dedicated to covering the cardiac catheterisation lab)6 was measured by report from a single respondent at each hospital at 0, 12 and 24 months. To quantify uptake of strategies, we used binary survey items, completed by the hospital primary contact person, indicating whether or not the hospital had implemented each strategy. To examine trends in RSMR before and during the intervention, we obtained RSMR values for each participating hospital from CMS Hospital Compare. CMS reports 3-year averages in RSMR; thus, we examined changes in hospital RSMRs between July 2011 and June 2014 (the period immediately preceding the intervention) and July 2012 and June 2015, the most contemporary data available. We also examined RSMRs from the July 2010 to June 2013 period to understand subsequent changes in the context of longer trends.
We collected qualitative data using in-depth, in-person interviews32 at baseline and at 6 and 18 months, with staff purposefully selected for diversity of roles from among the guiding coalition as well as other hospital executives, using a standardised interview guide (online supplementary appendix C). Interviews were approximately 45 min in duration, were audiotaped and professionally transcribed. We conducted ethnographic observations39 at baseline (n=40 hours) and 18-month (n=16 hours) visits using a standardised observation guide (online supplementary appendix D). Given the negligible risks for participation, the study was determined to be exempt from institutional review board review; all study participants provided their verbal informed consent to participate in this study.
Data analysis
We used standard frequency analysis to describe the samples of hospitals and survey respondents. We conducted descriptive analyses of quantitative data pertaining to organisational culture (overall and for five subscales) as well as uptake of evidence-based strategies reported at baseline, 12 months and 24 months. To assess quantitative changes in organisational culture overall and each domain over time, we used hierarchical generalised linear modelling to estimate the association between mean culture scores (overall and culture subscales) and time, accounting for clustering of individual respondents within hospitals. Analyses were corroborated independently by two analysts and conducted using SAS V.9.4.
To enhance the assessment of culture change with greater validity than possible with only quantitative instruments, we also used qualitative data,29 40analysed by a six-member multidisciplinary team using the constant comparison method of analysis.41 Each transcript was coded independently by at least three analysts, with discrepancies reconciled through negotiated consensus. Iterative coding and analysis occurred across each wave of data collection, with refinement and review by the full team of six analysts, until a final code structure was established and reapplied to the full data set.42 The qualitative analyses were performed with all analysts blinded to the quantitative results across all three waves of analysis; unblinding to both survey results and RSMR occurred when data were merged for final classification as described below. Hospitals were classified into two groups: hospitals that experienced substantial culture change (n=6) and hospitals that did not (n=4) based on meeting either quantitative or qualitative criteria.43 The quantitative criterion was a statistically significant change in the overall culture score between baseline and 2-year follow-up. The qualitative criterion was a marked shift in organisational culture, as reported by hospital staff through in-depth interviews from baseline to 2-year follow-up. Hospitals with a marked shift in culture were characterised by substantial, consistent, specific illustrations of notable changes in three to five of the domains of culture. In all hospitals classified as having a marked shift, there were no individuals who described lack of change or provided examples of persistent negative aspects of culture.
Two hospitals (IDs A and I) experienced statistically significant as well as marked qualitative shifts in culture, and four additional hospitals (IDs C, F, G and J) experienced marked qualitative shifts, although quantitative changes did not reach statistical significance. The four remaining hospitals (IDs B, D, E and H) experienced neither statistically significant nor marked qualitative changes in culture. We used comparative analysis32 to characterise the differences between the six hospitals that achieved substantial culture change and those that did not, and we compared changes in mean RSMR among these two groups of hospitals using t-tests. Qualitative analyses were performed with ATLAS.ti (Berlin, Germany) software.
Research team and reflexivity
Our research team was diverse with regard to disciplinary background, training and expertise. Most, although not all members, have over a decade of experience in studying quality of cardiovascular care. We did not have relationships with participants prior to the study, and our motivations for conducting the research were described during the informed consent process.
Results
Study hospitals and respondents
Study hospital (n=10) characteristics include census region (South 30%, Northeast 10%, Midwest 40%, West 20%), teaching status (teaching 20%, non-teaching 80%), beds (100–299, 20%; 300–499, 30%; 500+, 50%) and AMI cases per year (200–399, 50%; 400–599, 20%; 600–799, 30%). Participant roles are shown in table 1; survey response rates at baseline, 12-month and 24-month waves were 88% (147/168), 83% (154/186) and 94% (167/178), respectively, with 223 individual respondents (average response rate 88%). We conducted 393 in-depth interviews at baseline (n=162), 6 months (n=118) and 18 months (n=113), with a total of 197 individual respondents.
Quantitative results
In the full sample of 10 hospitals, we found a statistically significant change (p<0.05) in the overall measure of organisational culture and processes between baseline and 24 months (figure 1). The magnitude and depth of changes, however, varied substantially across hospitals. The use of evidence-based strategies also increased significantly over the 2-year study period (from a per-hospital average of 2.4 strategies at baseline to 3.9 strategies at 24 months; p<0.05), with most changes occurring between baseline and 12 months (table 2). Between the 2011–2014 and 2012–2015 reporting periods, the six hospitals that experienced substantial culture shifts showed significantly greater decreases in mean RSMR compared with changes in mean RSMR among the four hospitals that did not and compared with changes in mean RSMR nationally (figure 2 and see also online supplementary appendix E). Changes in hospital RSMRs prior to the LSL launch (2010–2013 to 2011–2014) did not differ significantly between the six hospitals that experienced substantial culture change and the four that did not, or between either group and the national average (p>0.05). Among the six hospitals that had substantial culture shifts, the RSMR decreased significantly from 2011–2014 to 2012–2015 (mean difference 1.07, p value for paired t-test p=0.003), while among the four hospitals without substantial culture change the mean difference was not significant (mean difference for four hospitals was 0.23, p=0.40 for paired t-test).
Qualitative results
For the six hospitals that experienced substantial positive culture change, changes were most prominent in three domains of culture (learning environment, senior management support and psychological safety). Differences between the six hospitals with positive culture change and the four hospitals without marked change are synthesised below; illustrative quotations appear in table 3.
Changes in learning environment
In the six hospitals that demonstrated positive shifts in culture, changes in the learning environment included increased use of data to drive change, greater creativity, enhanced problem-solving capacity and more frequent review of progress. Through working on the coalition, members became more reflective, which ultimately helped them develop a shared understanding of problems. One Director of Cardiovascular Quality noted that, through working on the coalition, members were eager to dig deeper for root causes: “We were able to pull away and think, ‘wait, there is opportunity here. How do we make a conscious effort to really push beyond just the surface level?’” (ID C_10). Participants grew increasingly open to new ideas, engaging front-line staff and enhancing the coalition’s ability to solve problems creatively. Participants described overcoming habitual ways of thinking and finding novel approaches to persistent problems, as this Director of Emergency Services reported a shift in their mindset over time: “We don’t need to keep doing things how we’ve always done them. We all know that’s the definition of insanity, doing the same thing and expecting different results. [LSL] has helped all of us realize that we need to be creative” (ID I_03). Coalitions demonstrated greater ownership of their data in both measuring progress and communicating data in credible ways to other clinicians. One Quality Director described how clinicians began to take an active interest in their performance data: “Within the last three months, all of the LSL data we’ve been collecting is now funneling into [cath lab committee]. That’s a really big change. The cardiologists have requested to be involved and to see more things now” (ID I_12).
In contrast, participants in the four hospitals without measurable culture change described persistent undervaluing of quality data, limited capacity for creative problem solving, lack of shared responsibility for solving problems, and aversion to experimentation and risk-taking. One Chief Nurse for Quality reflected: “unfortunately, we are not…even remotely in a place where creative problem solving is acceptable. We’re pretty much given templates to work from. We do not see at all a whole lot of opportunity to creatively problem solve” (ID E_11).
Opportunities for creativity were constrained by deference to hierarchical relationships; non-physician staff yielded too readily to physicians and physicians showed limited respect for diverse expertise.
Changes in senior management support
In the six hospitals that experienced a positive shift in culture, increases in senior management support for improving AMI care were manifest in several ways: greater senior management engagement and visibility; increased responsiveness and support for improvement efforts; empowerment of middle managers and the front line; and higher levels of accountability. Participants noted that senior management were more accessible than with prior improvement efforts, promptly allocating human and financial resources when needed. As one nurse manager remarked, “Because of administration being on LSL, they saw the importance, so they allocated money…When you have the right people at the table…we got things done quicker” (ID A-17). As coalitions became more experienced, senior management empowered both middle managers and front-line staff to design and implement interventions. As a Director of Cardiac Services described his personal development through the project, “my ability to let [the managers] be independent and come back with a recommendation is different. That is new for me” (ID D_11). Staff also described increasing accountability, some of which was painful but also promoted progress. One participant reflected on a transformational moment when the Chief Executive Officer delivered a ‘Yogi Berra speech’ to the coalition for their lack of progress; staff recounted this act as highly motivational, galvanising their shared sense of responsibility for improvement.
In the four hospitals without measurable culture changes, senior management support was perceived as persistently low over the 2-year period. Participants described frustration with senior management expressing support for LSL yet failing to provide requested resources or to address persistent obstacles. Coalition members expressed concern over disengagement, particularly of physicians, such as this Director of a Heart and Vascular Center:
"One of the biggest changes that I’ve seen with this coalition is the willingness to stick with it…If we lose our physician representative, we’re going to get someone new in that has no idea of the history. We need good leadership to stay dedicated to what we started, and that’s something that I worry about" (ID E_12).
The sporadic presence or complete absence of senior management at meetings made it difficult for coalition members to remain engaged and believing that the initiative was a priority for the organisation. As one Director of Organizational Quality reflected, “We have not had strong executive ownership in this project at all…that really changes the dynamics and the purpose of why we’re there” (ID E_11).
Changes in psychological safety
Although not statistically significant in the quantitative data, changes in psychological safety in the six hospitals with positive shifts were strongly apparent in the qualitative data and palpable to staff, who described increased freedom to voice concerns, greater respect across disciplines and departments, and greater appreciation for diverse expertise that supported productive collaboration. Coalition members reported developing higher confidence to express divergent views, particularly those that challenged powerful roles in the hospital. Physicians and non-physicians alike commented on the levelling effect of working together in the coalition, with more equitable participation and engagement among members, who grew more unified as a team. In one hospital, the coalition set a new tone for risk-taking and working on the ‘leading edge’, even if some ideas were not successful. One participant highlighted a transformation in their hospital, in which staff had traditionally avoided confrontation: “Sometimes we’re just so nice we won’t talk about the hard things. So what I have seen lately is we’re still very nice but we’re becoming a little more up front in our communication. That to me has been super exciting to see” (ID I_14).
In contrast, participants in the four hospitals without measurable culture change described how their improvement efforts stagnated despite participation in LSL. Participants reported difficulties in fostering trust in others, and the ability to speak freely was not widely shared. As one cardiologist observed, “We are very open among cardiology leadership. Now, you could say, at the staff level, not so much. I think there are some opportunities there” (ID H_7). Decision making was described as opaque and staff reported “walking on egg shells” as they tried to implement changes. In these hospitals, staff remained focused on their own individual or departmental interests, rather than collaborating to solve problems. One cardiologist reflected on his experience of coalition meetings as pro forma, where reports of progress on improvement projects might be influenced by the dynamics between managers and their bosses: “There’s a lot of middle managers that are covering their butts…It doesn’t necessarily mean a problem has been solved” (ID B_15). Coalitions in these hospitals faced challenges in communication and information sharing that constrained the team’s ability to function effectively. One coalition member described difficult dynamics in the team that persisted until another member “took the risk of being vulnerable” to raise the issue. He observed that “in this work environment…sometimes comments are filtered” (ID H_13).
Discussion
In this intervention designed to promote positive change in organisational culture, we observed substantial shifts in culture, driven largely by an improved learning environment, increased senior management support and increased psychological safety, through a team-based model of clinical leadership. This finding is consistent with prior research demonstrating the importance of engaging physicians and non-physicians, as well as senior leaders and middle managers, in a learning environment16 44; our study further documented significant changes in RSMR, a meaningful clinical outcome.45 Hospitals that experienced substantial culture shifts also achieved significantly greater reductions in RSMR than those that did not experience such culture shifts or than hospitals nationally.
To our knowledge, this is the first prospective interventional study to demonstrate improvements in organisational culture and reductions in RSMR for patients with AMI. The longitudinal data indicate that RSMR was decreasing before LSL both in hospitals with substantial culture change and in those without substantial change; this trend of decreasing RSMR was maintained during LSL in the six hospitals with substantial culture change, whereas progress plateaued in the four hospitals that did not experience marked culture change, as it did nationally. Moreover, although the quantitative magnitude of changes in culture was relatively modest, the qualitative experiences were compelling, a finding we attribute in part to the difficulty of quantitative measurement of nuanced concepts such as culture.24 25 46 47 Experts have cautioned that although culture is important to hospital performance, change can be very difficult and take up to a decade.22 46 Our findings suggest that, despite these concerns, organisational culture can be modified through development and support of a multidisciplinary leadership group that establishes a shared goal with senior management and uses data effectively to drive change, ways of working that experts have identified as essential for improvement.48 49
Staff in hospitals that experienced substantial culture change described plans to ‘transfer LSL’ to other improvement efforts (eg, heart failure readmissions and population health). This intention for replication suggested that changes were related to norms of work and culture, rather than more superficial, technical improvements in AMI care. Researchers have lamented the difficulty of measuring nuanced concepts such as culture quantitatively24 25 46 47; however, we were able to detect gradations in culture and to produce insights about the differential effects of the intervention, thus highlighting the benefits of applying mixed methods to studies of organisational change and clinical performance.30 40
Our findings should be interpreted in light of several limitations. First, the lack of randomisation or comparison group limits our ability to make conclusions about causality. Nevertheless, a randomised controlled trial would have limited the depth and breadth of contextual factors; failure to include context as a key variable in implementation efforts is a notable limitation of research to date.21 24 28 50 Our longitudinal, convergent mixed-methods intervention design was well-suited for studying complex change processes30 51 in a set of diverse hospitals. Second, participants may have had a vested interest in representing the intervention as a success; to minimise social desirability bias, we interviewed multiple staff in each hospital, elicited details that would be difficult to misrepresent, encouraged respondents to share both positive and negative experiences, and triangulated across multiple data sources.40 52 Third, RSMR is reported by CMS in 3-year blocks, so we could not measure annual changes precisely; nevertheless, we were able to detect significant differences between hospitals in changes in RSMR. Last, measurement of both culture and RSMR was limited to within the intervention period; although we detected significant changes in culture and RSMR, longer follow-up would provide greater understanding about how the intervention effects change over time. In addition, the increment of quantitative change in culture was modest; because this is a newly validated instrument,38 we are unable to interpret the magnitude of change relative to other studies. Nevertheless, a strength of mixed-methods designs is the complementary qualitative data; the combination of statistically significant changes in the quantitative measure coupled with strong qualitative evidence together indicates substantive changes in culture. We would expect, however, that larger culture changes would take place over a longer follow-up period, given that culture emerges through shared experience and over time.46
We found that the LSL intervention was effective for 6 out of 10 hospitals in fostering positive shifts in hospital organisational culture. Furthermore, hospitals with marked improvements in culture experienced a significantly greater decrease in RSMR than those that did not improve culture. Our study addresses a critical gap in the performance improvement literature, which has highlighted the importance of hospital culture for clinical outcomes but has not demonstrated whether or how hospital culture can be influenced to improve clinical outcomes. Findings may be useful to clinicians and hospital leadership seeking to promote improvements in care and outcomes for patients with AMI.
Acknowledgments
The authors thank David Hayes, MD, Victor Montori, M.D, the Mayo Clinic and the independent health organizations within the Mayo Clinic Care Network for their collaboration. We are grateful to Richard Kovacs, MD, Joe Drozda, MD and the American College of Cardiology for their active participation in hospital workshops. Finally, we thank each hospital and their LSL guiding coalitions for their time and dedication.
References
Footnotes
Contributors LAC, EL and EHB conceptualised, designed and obtained funding for the study. LAC is the Principal Investigator. EHB, AB, EL and LAC drafted the manuscript. All authors critically revised the manuscript for important intellectual content and approved the final version.
Funding Funding was provided through a research grant from The Medicines Company, Parsippany, NJ.
Competing interests None declared.
Ethics approval All research procedures were approved by the Human Investigation Committee at the (intentionally omitted for blind review) Human Research Protection Program Committee at (intentionally omitted for blind review) School of Medicine.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Requests for additional unpublished data from interested researchers will be carefully considered on an individual basis upon written request to the corresponding author. However, study participants consented to participate in the study under a guarantee of confidentiality. Because of the small sample of hospitals drawn from a defined network and the specificity of clinical roles and experiences within hospitals, adequate de-identification of unpublished qualitative and quantitative data is likely not possible.