Article Text
PDF
Abstract
Background Quality collaboratives are widely endorsed as a potentially effective method for translating and spreading best practices for acute myocardial infarction (AMI) care. Nevertheless, hospital success in improving performance through participation in collaboratives varies markedly. We sought to understand what distinguished hospitals that succeeded in shifting culture and reducing 30-day risk-standardised mortality rate (RSMR) after AMI through their participation in the Leadership Saves Lives (LSL) collaborative.
Procedures We conducted a longitudinal, mixed methods intervention study of 10 hospitals over a 2-year period; data included surveys of 223 individuals (response rates 83%–94% depending on wave) and 393 in-depth interviews with clinical and management staff most engaged with the LSL intervention in the 10 hospitals. We measured change in culture and RSMR, and key aspects of working related to team membership, turnover, level of participation and approaches to conflict management.
Main findings The six hospitals that experienced substantial culture change and greater reductions in RSMR demonstrated distinctions in: (1) effective inclusion of staff from different disciplines and levels in the organisational hierarchy in the team guiding improvement efforts (referred to as the ‘guiding coalition’ in each hospital); (2) authentic participation in the work of the guiding coalition; and (3) distinct patterns of managing conflict. Guiding coalition size and turnover were not associated with success (p values>0.05). In the six hospitals that experienced substantial positive culture change, staff indicated that the LSL learnings were already being applied to other improvement efforts.
Principal conclusions Hospitals that were most successful in a national quality collaborative to shift hospital culture and reduce RSMR showed distinct patterns in membership diversity, authentic participation and capacity for conflict management.
- Quality improvement
- Teams
- Teamwork
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Introduction
Quality collaboratives and campaigns are widely endorsed by researchers and policymakers as a potentially effective method1 for translating and spreading best practices for acute myocardial infarction (AMI) care. Such efforts convene hospitals to share experiences, promote adoption of evidence-based practices, and foster systematic approaches to quality improvement.2 The last 15 years have seen a number of quality collaboratives focusing on AMI care, such as Get with the Guidelines3 and the Door-to-Balloon Alliance,4–9 and a concomitant reduction in AMI mortality. Based on recent evidence about the influence of hospital culture on AMI mortality,10–12 the Leadership Saves Lives (LSL) quality collaborative13 was a national effort of 10 hospitals to improve hospital culture and reduce 30-day risk-standardised mortality rate (RSMR) after AMI over 2 years.13
Although quality collaboratives have been associated with hospital improvements, not all hospitals that participate in collaboratives experience substantial gains, and some continue to lag despite similar exposure to collaborative interventions.2 14–17 Two features distinguish hospitals that have experienced larger performance gains through their participation in quality collaboratives. The first of these is strong senior management support for improvement efforts.18–25 Studies highlight specific behaviours by senior management that seem to matter, which include but are not limited to: maintaining visibility in improvement efforts, connecting efforts to the organisational mission, engaging front-line staff, providing necessary resources and establishing accountability for performance.18–25 The second feature that has been found to distinguish higher performing hospitals within quality collaboratives is teamwork.16 17 22 26 27 These studies indicate that teams in hospitals that achieve the goals of quality collaboratives are more likely to know and respect each other, have stable membership and have effective team leadership.28–35 Although this literature is useful, we could find no longitudinal studies with rich qualitative and quantitative data that have examined how hospitals develop (or fail to develop) these capacities within the context of a quality collaborative. Understanding this process of development could enhance the potential for hospitals to maximise the positive impact of participation in quality collaboratives.
Accordingly, we sought to understand what distinguished hospitals that succeeded to shift culture substantially and to reduce 30-day RSMR after AMI through participation in the LSL collaborative. We focused on the experience of the multidisciplinary team created to guide the collaborative efforts in each hospital; we refer to this team as the hospital ‘guiding coalition,’ using the term introduced as part of the National Demonstration in Quality Improvement36 and used broadly in the change management literature.37 38 As part of the collaborative, each hospital was asked to appoint a guiding coalition to guide improvement efforts, which included approximately 15 multidisciplinary staff involved in care of patients with AMI. The findings may be useful for hospitals seeking to maximise the impact of their participation in quality collaboratives, and to healthcare professionals seeking to design successful improvement efforts.
Methods
Study design and sample
We used quantitative and qualitative data from a longitudinal, convergent mixed methods interventional study39 40 in which 10 of 12 hospitals approached (participation rate 83%; table 1) participated in a quality collaborative to improve aspects of organisational culture shown to be associated with lower 30-day RSMR after AMI. Hospitals were selected using random sampling with a purposive component,41 begining with a randomised list of sampling units but may omit units that are too similar to previously selected units, to ensure a diverse sample. We drew from members of the Mayo Clinic Care Network (MCCN), a national group of regional medical systems committed to quality improvement through collaboration. From the 21 MCCN members as of January 2014, we identified those meeting eligibility criteria, including: (1) at least 200 AMI discharges per year in order to ensure depth of experience in caring for patients with AMI; (2) for multihospital systems, the largest hospital in the system; and (3) 30-day RSMR after AMI that were at or above the national median between 2009 and 2012 to indicate room for improvement. We randomised the list of eligible hospitals (n=18), and beginning from the top, worked in sequence down the list to recruit 10 hospitals that were diverse in teaching status and geographic region (based on US Census categories). Across the participating hospitals, we surveyed 223 individuals (table 2). Survey response rates at baseline, 12-month and 24-month waves were 88% (147/168), 83% (154/186) and 94% (167/178), respectively. We also conducted 393 in-depth interviews at baseline (n=162), 6 months (n=118) and 18 months (n=113) with a total of 197 individual respondents (table 2).
Hospital characteristics (n=10 hospitals)
Participant characteristics
The LSL intervention
The 2-year intervention, previously described in detail,13 was designed to foster key dimensions of organisational culture that are relevant to hospital performance (eg, learning environment, psychological safety, commitment to the organisation, senior leadership support and time for improvement efforts).10 42 43 Each hospital was asked to appoint a guiding coalition that included approximately 15 key staff involved in care of patients with AMI. Coalition members included staff from multiple departments (eg, cardiology, emergency medicine, pharmacy, quality improvement, cardiac rehabilitation), professions (eg, physicians, nurses, technologists, administrators, physician assistants) and levels of the organisation (from senior executives to front-line staff).
In each hospital, all guiding coalition members participated in four 1-day, on-site workshops during the study period, and a subset of 4 of these members participated in three all-hospital annual forums to promote sharing of experiences across hospitals. The workshop curriculum (see online supplementary appendix A) was grounded in a strategic problem-solving approach44 in which coalitions sought to foster organisational culture to promote better performance as they implemented evidence-based strategies associated with lower RSMRs.11 Networking among hospitals and access to related programme materials were facilitated through a web-based platform (Basecamp Software V.3; Chicago, Illinois).
Supplemental material
Data collection and measures
We assessed organisational culture using a validated, 31-item close-ended instrument (see online supplementary appendix B) consisting of five domains: learning and problem solving, psychological safety, commitment to the organisation, senior leadership support and time for improvement efforts. The survey was administered to all guiding coalition members at baseline, 12 months and 24 months. In addition, we used a standard discussion guide (see online supplementary appendix C) to conduct in-person, in-depth interviews41 of guiding coalition members at baseline, 6 months and 18 months. Lasting on average an hour, the interviews generated extensive qualitative data. We also conducted 56 hours of selective observations of key interactions in care for patients with AMI (eg, patient rounds, relevant meetings and other on-site activities) at baseline (40 hours) and 18 months (16 hours), and took field notes during these observations. All research procedures were approved by the Human Investigation Committee at the Yale Human Research Protection Program at Yale School of Medicine. Given the negligible risks for participation, the study was determined to be exempt from Institutional Review Board review; all study participants provided their verbal informed consent to participate in this study.
Supplemental material
Supplemental material
Data analysis
Hospitals were deemed to have experienced substantial culture shift if they met at least one of two criteria. As recommended in convergent, mixed methods studies,39 40 the criteria were quantitative and qualitative. The quantitative criterion was a statistically significant change in the overall culture score between baseline and 2-year follow-up using a 95% confidence level for significance. The qualitative criterion was a marked shift in culture within domains previously validated45: (1) learning and problem solving; (2) senior leadership support; (3) psychological safety; (4) commitment to the organisation; and (5) time for improvement, and as expressed by hospital staff through in-depth interviews from baseline to 2 years of follow-up. Two hospitals (IDs A and I) experienced statistically significant and marked qualitative shifts in culture, and four additional hospitals (IDs C, F, G and J) experienced marked qualitative shifts although the quantitative change did not reach statistical significance. The remaining four hospitals (IDs B, D, E and H) experienced neither statistically significant nor marked qualitative changes in culture. Thus, we compared the experiences of the six hospitals that experienced substantial culture change as measured quantitatively or qualitatively with the four hospitals that did not.
To examine trends in RSMR before and during the intervention,we obtained RSMR values for each participating hospital from CMS Hospital Compare. CMS reports 3-year averages in RSMR; thus, we examined changes in hospital RSMRs between July 2011 and June 2014(the period immediately preceding the intervention)and July 2012 and June 2015, the most contemporary data available. We also examined RSMRs from theJuly 2010 to June 2013 period to understand subsequent changes in the context of longer trends. Between the 2011–2014 and 2012–2015 reporting periods, the six hospitals that experienced substantial culture shifts showed significantly greater decreases in mean RSMR compared with changes in mean RSMR among the four hospitals that did not and compared with changes in mean RSMR nationally. Changes in hospital RSMRs prior to the LSL launch (2010–2013 to 2011–2014) did not differ significantly between the six hospitals that experienced substantial culture change and the four that did not, or between either group and the national average (p>0.05). Among the six hospitals that had substantial culture shifts, the RSMR decreased significantly from 2011–2014 to 2012–2015 (mean difference 1.07, p value for paired t-test p=0.003), while among the four hospitals without substantial culture change the mean difference was not significant (mean difference for four hospitals was 0.23, p=0.40 for paired t-test).46
In addition to the qualitative analysis, we calculated t-tests to compare average coalition sizes and turnover rates between the six hospitals that achieved substantial positive culture change, as determined by quantitative and qualitative data (hospitals A, C, F, G, I and J) and the four hospitals that did not (hospitals B, D, E and H). Using a confidence level of 95%, we tested the hypotheses that hospitals with larger guiding coalitions with less turnover would be more likely to achieve substantial culture shifts through their participation in LSL.
We examined recurrent themes in the development and work of the guiding coalitions using the constant comparative method41 47 48 applied by a six-member multidisciplinary research team. We compared the recurrent themes as expressed by staff in the six hospitals with substantial culture shift versus in the four hospitals without culture change. We used ATLAS.ti (Berlin, Germany) to facilitate analysis. As recommended by experts in qualitative data analysis,41 49 we searched for disconfirming evidence to enhance credibility of findings,50 used triangulation with multiple sources of data to enhance validity,51 and kept a detailed audit trail to document analytical decisions.
Results
Overview
The experience of the guiding coalitions, all of which were exposed to the same intervention protocol over the same period of time, varied markedly across the 10 hospitals. In the six hospitals that demonstrated substantial positive shifts in culture, we found (1) representation of staff from different disciplines and levels in the organisational hierarchy; (2) authentic participation and engagement of diverse perspectives in the work of the guiding coalition; and (3) distinct patterns of managing conflict, fatigue and motivation over time. Participants from the remaining four hospitals, which did not experience shifts in culture, experienced challenges in recruiting and retaining diverse staff to the guiding coalition, experienced superficial participation of staff who did attend, and had difficulty managing conflict, blame or boredom within the guiding coalition. Below we elaborate these themes describing the contrasts between experiences in the six hospitals that achieved culture shifts compared with the four hospitals that did not.
Membership in the guiding coalition
Neither the size nor the turnover rates of the guiding coalition members varied significantly between the six hospitals with culture shifts and the four without (average size 21 vs 18 members, p value 0.21; average turnover rate 26% vs 22% of starting guiding coalition, p value 0.50, respectively). Nevertheless, the guiding coalitions in the six hospitals that experienced substantial culture shifts had starkly different experiences than the guiding coalitions did in the four hospitals without culture change.
In the six hospitals that had substantial positive culture shifts, coalitions achieved diverse membership, both across departmental lines but also including both front-line, mid-level, and top leadership clinical and administrative staff. As they undertook efforts to implement evidence-based practices, the coalitions recognised gaps in their membership and addressed them. For instance, in the course of their work, one hospital noted they had omitted case management from the original group. Recognising that posthospital care was central to reducing 30-day mortality, they promptly added a case manager. At the same time, in cases where members became disengaged and appeared not to be contributing, coalitions allowed for turnover with replacement as needed. In general, staff in these hospitals understood that diverse membership was critical for both discovering the root causes of problems and fostering team ownership of the problem and potential solutions, as illustrated by one nurse,
“That was the ‘ah-hah’ moment, where we knew—if we’re going to do this root cause analysis, we need to have the right people in the room. So we had a meeting with all the right stakeholders and people being able to walk out of here owning what they can do.” (IDI_3, Nurse)
This perspective was in stark contrast to experiences in the four hospitals that did not experience culture change. In these hospitals, staff described having difficulty constructing adequately diverse guiding coalitions. Some included nursing without recognising diversity within nursing (eg, nurse managers vs staff nurses, or nurses on patient care units vs those in the catheterisation laboratory). Other hospitals split the guiding coalition into smaller groups to increase efficiency but then were unable to capitalise on the diversity of the larger group. One quality manager described,
“It’s been kinda painful getting the group to coalesce, and getting traction. Here we are a year into it and I feel like we just started.” (IDH_1, Quality Management Staff)
Participation by members
In the six hospitals with substantial positive culture change, participation in the guiding coalition was marked by discovery, learning, and teaching among and between members, and across staff who operated at different levels in the hospital hierarchy. A paramedic enthusiastically reported,
“‘Every time I come to a meeting, I learn something.’ Reflecting further, he said, ‘and they ask me for my input. I am not just a fly on the wall here. I appreciate that’.” (IDA_1, Paramedic)
Staff across these hospitals remarked that participation was authentic, which was described as being clear about expectations and performance. One cardiologist noted,
“‘I found that as a group, we had lots of talents that suddenly got brought to bear that I didn’t even know were there.’ People that had time and energy to do it all of a sudden just jumped into the fray and started working.” (IDJ_3, Cardiologist)
High levels of participation did not commence instantly; rather it emerged over time as members began to offer their unique skills and perspectives to the shared task. An emergency medicine physician described this evolution,
“I think [we] transitioned from everyone telling everyone else what to do to looking at themselves and saying this is what we’re going to do make this work. That was really a turning point.” (IDC_8, Emergency Medicine Physician)
In contrast, staff from the four hospitals that did not experience culture shifts described the work falling to a single department, typically quality management. Coalition members described being in attendance but ‘quiet’ and not authentically participating in the work of the group. At different hospitals, staff said,
“We were participating because somebody wanted us to. Somebody higher up said, ‘You’re going to participate’.” (IDD_14, Quality Management Staff)
“I feel like we haven’t yet gotten to the meat of the matter. I feel like we’re lost. We’re creating lots of pretty things to look at and totally missing the point.” (IDH_7, Nurse)
Managing conflict, fatigue and engagement
Managing conflict, fatigue and engagement was a universal experience shared across all intervention hospitals. The guiding coalitions in all 10 hospitals faced moments of conflict among their membership. In some hospitals, conflict was expressed in blaming behaviours, across departmental lines (eg, the emergency department and the catheterisation laboratory) and across hierarchy (eg, front-line clinical and clinical management staff). In other hospitals, staff reflected more passive manifestations of conflict, such as ‘falling asleep when someone else was talking’, ‘getting people off track’ or obfuscating the issues.
Prominent in six hospitals with substantial positive culture change and largely absent in the four hospitals that did not experience culture change was the use of an array of tactics for managing conflict and sustaining genuine engagement. Staff described being cognisant of not wasting people’s time, having clear roles and defining work to be delivered between meetings. Some groups broke into subgroups for specific tasks but continued to link to the full coalition. Additionally, members of these coalitions reported they were careful to be sure opinions were not ignored while still keeping the group ‘on task.’ Staff reported that revisiting of the larger goal of improving AMI care helped align and reinvigorate staff when they became distracted or overburdened. One Vice President described refocusing the coalition,
“The other thing that we said is, ‘It’s not just about us…It’s about population health, chronic disease management, how we take care of people when they’re at home.…We know we can’t do that without really high-level collaboration’.” (IDF_15, Vice President)
In the four hospitals that did not experience culture change, coalition members described an environment in which attendance at agreed-upon meetings was unreliable and blaming behaviours persisted. One Chief Medical Officer commented on lack of trust among the coalition,
“The cardiologists tell me that it’s the emergency room’s (ER’s) fault because they don’t know how to read EKGs. When, in fact, the cardiologists have said, ‘We don’t want the activation from the field. There’s a lot of incongruity between what the cardiologists are saying and what they’re actually doing [and between] what the ER is saying and what they’re actually doing. It’s not a real trusting environment’.” (IDB_9, Chief Medical Officer)
Views on future use of LSL learnings
Staff estimations of the likelihood of applying the learnings from LSL to other projects and areas in the hospital varied substantially. Staff from guiding coalitions in the four hospitals that did not make noticeable culture changes were less certain that the LSL approach would continue, citing that few people understood the work of the guiding coalition and that it had been difficult to demonstrate its ‘return on investment.’ In contrast, staff in the six hospitals that experience marked culture shifts were optimistic about ‘ingraining’ the LSL approach into other areas. For example, staff reported that the learnings from the guiding coalition approach might be applied to improve the quality of care for patients with complex conditions such as heart failure and stroke care, reduce unplanned readmission rates, and achieve population health goals.
Discussion
We identified key aspects of guiding coalition membership, participation and conflict management capacities that were prominent in the six hospitals that achieved positive culture change and absent in the four hospitals that did not. Guiding coalition members in the six hospitals with substantial positive culture change represented multiple disciplines and departments and worked at different levels of the hospital hierarchy, or even outside the hospital (such as emergency medical services or private cardiologists). Furthermore, they were successful in fostering meaningful participation by the members. Last, they were able to manage conflict in ways that energised and sustained the work of the coalition. Although other studies have suggested that team size and turnover may influence effectiveness,27 52–54 in this study with rich qualitative data, we were able to identify nuanced aspects of both inputs and processes, as suggested by West and Lyubovnikova,55 of guiding coalitions that may explain their differential success.
Our study extends the previous literature by identifying concrete characteristics and actions guiding coalitions implement that maximise the impact of teamwork and senior management support on culture. Our results also indicate that turnover is not tantamount to failure. Rather, successful coalitions built a way of working together that conferred resiliency in the face of turnover and resource challenges. Importantly, both teamwork and leadership from senior management were fundamentally interwoven in the work of successful guiding coalitions. Also critical to the guiding coalitions in the successful hospitals was the sense of learning from others who had different experiences and perspectives but were committed to the shared goal. Previous research has also identified ‘learning organizations’ as those that can withstand shifts in marketplaces and technologies56 57; our study demonstrated the importance of membership, participation and conflict management capabilities to enable such a learning environment, which allows groups to solve problems effectively.58–60
Our findings should be understood in light of some limitations. First, the LSL intervention was not designed as a randomised controlled trial, which limited our ability to make statistical inferences. It is possible that unmeasured factors in the context of the six hospitals and not apparent in the four might explain our findings. Nevertheless, our longitudinal, convergent mixed methods intervention design39 40 was well suited to understanding how guiding coalitions worked within their contexts, the central phenomenon of interest. Throughout, we applied rigorous methods of in-depth inquiry, observation and qualitative data analysis to mitigate effects of omitted data or bias. Second, given the in-depth nature of the intervention,13 we focused on 10 hospitals, and although they were diverse in terms of geographical location and size, results may differ in other settings. Last, it is possible that the findings were influenced by the Hawthorne effect61 in which the act of observing changes behaviours of the observed or that social desirability bias62 occurred. As others have shown,63 measuring organisational culture is highly complex. Nevertheless, we worked to mitigate these biases by conducting both observations and interviews over time to triangulate what we were told. Additionally, we applied multiple methods to limit bias including multiple interviewers and analyst from varying backgrounds, a consistent protocol for data collection and analysis, and a detailed audit trail to document analytical decisions.
In conclusion, although previous work has documented the effectiveness of the LSL intervention on influencing the organisational culture and RSMR in 6 of the 10 hospitals, this study addressed the question of how the membership, participation and conflict management capacities of the guiding coalitions may have allowed for differential success across the 10 participating hospitals. The implication of these findings is that careful attention to the design and activities of such coalitions, together with developing capacity for managing inevitable challenges such as turnover and conflict, may help hospitals increase the benefits of participation in quality collaboratives.
Acknowledgments
We thank David Hayes, MD, Victor Montori, MD, the Mayo Clinic and the independent health organisations within the Mayo Clinic Care Network for their collaboration. We are grateful to Richard Kovacs, MD, Joe Drozda, MD, and the American College of Cardiology for their active participation in hospital workshops. Finally, we thank each hospital and their LSL guiding coalitions for their time and dedication.
References
Footnotes
Contributors LAC, EL and EHB conceptualised, designed and obtained funding for the study. LAC is the principal investigator. EHB, AB, EL and LAC drafted the manuscript. All authors critically revised the manuscript for important intellectual content and approved the final version.
Funding Funding was provided through a research grant from The Medicines Company, Parsippany, New Jersey, USA.
Competing interests None declared.
Ethics approval Yale Human Research Protection Program; Human Investigation Committee.
Provenance and peer review Not commissioned; externally peer reviewed.