Article Text

other Versions

PDF

Project JOINTS: What factors affect bundle adoption in a voluntary quality improvement campaign?
  1. Dmitry Khodyakov1,
  2. M Susan Ridgely1,
  3. Christina Huang1,
  4. Katherine O DeBartolo2,
  5. Melony E Sorbero3,
  6. Eric C Schneider4,5,6
  1. 1RAND Corporation, Santa Monica, California, USA
  2. 2Institute for Healthcare Improvement, Cambridge, Massachusetts, USA
  3. 3RAND Corporation, Pittsburgh, Pennsylvania, USA
  4. 4RAND Corporation, Boston, Massachusetts, USA
  5. 5Department of Health Policy and Management, Harvard School of Public Health, Boston, MA, USA
  6. 6Division of General Medicine and Primary Care, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
  1. Correspondence to Dr Dmitry Khodyakov, RAND Corporation, 1776 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138, USA; Dmitry_Khodyakov{at}rand.org

Abstract

Background Diffusion and adoption of effective evidence-based clinical practices can be slow, especially if complex changes are required to implement new practices.

Objective To examine how hospital adherence to quality improvement (QI) methods and hospital engagement with a large-scale QI campaign could facilitate the adoption of an enhanced prevention bundle designed to reduce surgical site infection (SSI) rates after orthopaedic surgery (hip and knee arthroplasty).

Methods We conducted telephone interviews with hospital QI leaders from 73 of the 109 hospitals (67% response rate) in five states that participated in Project JOINTS (Joining Organizations IN Tackling SSIs), a QI campaign run by Institute for Healthcare Improvement (IHI). Using QI methods grounded in the IHI Model for Improvement, this campaign encouraged hospitals to implement an enhanced SSI prevention bundle. Hospital QI leaders reported on their hospital's adherence to the Project JOINTS QI methods; their level of engagement with Project JOINTS activities; and adoption of the SSI prevention bundle components. Interview data were analysed quantitatively and qualitatively.

Results Both adherence to the QI methods and hospital engagement were positively associated with complete bundle adoption. Hospital engagement, especially the use of project materials and tools, was also positively associated with the initiation of and improved adherence to individual bundle components.

Conclusions Our findings suggest that greater adherence to the QI methods and active hospital engagement in a QI campaign facilitate adoption of evidence-based patient safety bundles in orthopaedic practice.

Statistics from Altmetric.com

Introduction

Although the field of implementation science strives to address the recognised gap between knowledge generated in clinical trials and routine clinical practice,1 ,2 research has demonstrated significant delays in the diffusion and adoption of effective, evidence-based clinical practices.3 ,4 Closing this gap still remains a challenge for many health systems.

Quality improvement (QI) collaboratives,5 such as Institute for Healthcare Improvement's (IHIs) Breakthrough Series in the USA,6,7 the Health Foundation's Safer Patients Initiative8–12 and Stroke 90:10 QI Collaborative13 in the UK, and Evidence 2 Excellence in Canada,14 ,15 have been used to accelerate the adoption of evidence-based care. A collaborative is a multiorganisational effort to design and promote the adoption of a multicomponent QI package.16 ,17 QI collaboratives focus on a prespecified topic, offer access to expert knowledge on clinical and QI topics, provide peer support that can help an individual hospital identify and implement best practices; rely on multidisciplinary implementation teams within hospitals, use a QI methodology and encourage participants to engage in a series of structured activities.18 Collaboratives tend to generate enthusiasm and energy among participating organisations to effect change.7

Although the QI collaborative has become a popular approach, participants often have to invest considerable time, effort and financial resources to achieve and sustain even small improvements.7 ,8 Because collaboratives are resource-intensive, they may be limited in scope and difficult to scaleup.8 Even though certain aspects of teamwork, data collection and participation in collaborative activities have been shown to increase the collaborative's chances for success,18 several recent systematic reviews suggest that effectiveness of collaboratives is variable and setting-dependent.17 While it might be desirable to remove costly components of a collaborative, the imprecise descriptions of the QI intervention components make it difficult to identify those components that affect provider performance and patient outcomes.19

To expedite the process of disseminating evidence-based practices and to increase their reach while decreasing the burden on participants, the IHI introduced the QI ‘campaign’—a new approach that uses a field infrastructure called the Rapid Spread Network comprising local (usually state-based) QI organisations, professional societies and hospital associations, among others, to recruit hospitals and disseminate campaign information. In campaigns, hospitals are not required to attend numerous face-to-face meetings and provide periodic updates on their implementation efforts and success, which are typical requirements of IHI collaboratives.20 Instead, participating hospitals are encouraged to participate in campaign activities remotely (primarily via webinar meetings and an email listserv) and to use educational materials posted on the campaign website. Moreover, participating hospitals are encouraged to use a multifaceted QI approach developed by IHI to motivate and support change. The most well-known IHI campaigns are the 100 000 Lives21 ,22 and 5 Million Lives campaigns,23 which were designed to reduce medical harm by promoting adoption of evidence-based practices. The effectiveness of campaigns as a QI approach, however, is uncertain. Despite the compelling theoretical basis, campaigns remain something of a ‘black box’ in that the mechanisms by which they actually produce improvement are not well described.

The Project JOINTS intervention

Surgical site infection (SSI) following hip and knee arthroplasty is devastating for both patients and their caregivers. Treatment of such infections may require additional surgery, removal of the prosthetic joint, replacing the prosthetic with a spacer device and prolonged systemic antibiotic therapy. Patients unnecessarily suffer from pain and typically experience impaired mobility during recovery. Second and third surgeries to revise the implant are painful, burdensome for the patient and caregivers, and costly.24 ,25

Funded by the US Department of Health and Human Services, Project JOINTS (Joining Organizations IN Tackling SSIs) was an IHI campaign to spread a bundle of evidence-based practices that can prevent SSI among patients undergoing hip and knee arthroplasty.26 For simplicity of messaging, IHI used the term ‘bundle’ to describe what was actually a combination of two SSI prevention bundles that involve two points of care (the preop clinic and the operating room).27 The resulting five-component enhanced SSI prevention bundle included three evidence-based practices that were not widely used prior to 2010:

  1. use an alcohol-containing antiseptic agent for preoperative skin preparation28 ,29 because the use of alcohol in addition to a long-acting antiseptic agent provides superior protection against SSIs;

  2. instruct patients to bathe or shower with chlorhexidine gluconate (CHG) soap for at least three days before surgery30–32 to reduce bacterial colonisation of the skin;

  3. screen patients for Staphylococcus aureus (SA) carriage and decolonise SA carriers with 5 days of intranasal mupirocin and at least three days of chlorhexidine soap prior to surgery33 ,34 because SA nasal colonisation has been shown to be associated with an increased risk for SSI;

and two established practices that had become usual care in many hospitals:

  1. appropriate use of prophylactic antibiotics;

  2. appropriate hair removal for the location and procedure.35

Building on previous IHI campaigns, Project JOINTS used a field infrastructure called the Rapid Spread Network (see figure 1) that consisted of three sets of actors: IHI; state organisations, such as Quality Improvement Organisations and state hospital associations, known in the parlance of the campaign as ‘nodes’; and hospitals.

Figure 1

Conceptual model for Project JOINTS. JOINTS, Joining Organizations IN Tackling SSIs; SSI, surgical site infection; IHI, Institute for Healthcare Improvement.

IHI established the agenda for Project JOINTS, provided recruitment materials designed for specialised audiences, disseminated a ‘How to Guide’ describing a tailored set of QI methods recommended for hospitals and orthopaedic practices, developed all Project JOINTS intervention materials, created and maintained a project website and email listserv, and offered a range of learning opportunities (including webinar calls, faculty consultations and town hall meetings, among others) to participating hospitals (see online supplementary appendix A for a description of Project JOINTS materials, tools and activities).

Nodes were responsible for raising awareness about Project JOINTS at the state and local levels, recruiting hospitals, disseminating information about the project and helping IHI organise on-site visits and town hall meetings, where participating hospital teams from the same geographical region could come together to openly and informally share their Project JOINTS experiences, learn from each other and get help from IHI faculty in a face-to-face setting. Nodes were not required to provide ‘hands-on’ assistance to participating hospitals, there was no required level of ‘node’ participation and some of these state-level nodes were more active than others in Project JOINTS for a variety of reasons, including ongoing participation in other local or national initiatives that might not have been aligned with Project JOINTS.

Participating Project JOINTS hospitals were granted access to a password-protected website maintained by IHI, which contained the Project JOINTS print and online materials about SSI prevention (including a ‘How to Guide’ that described the recommended Project JOINTS QI methods; evidence handouts for surgeons, patients and family members; and a ‘business case’ for SSI prevention, among others). Hospitals were also encouraged to participate in a variety of interactive Project JOINTS activities, including a webinar call series and a project listserv. To facilitate implementation of as many evidence-based practices as possible, hospitals were encouraged to use three elements of the Model for Improvement associated with successful QI projects (1) developing a QI plan, (2) putting in place a multidisciplinary implementation team and (3) conducting small tests of change.6 No funding was provided to hospitals to defray costs associated with their participation in this campaign.

Although there are many contextual factors that affect QI intervention uptake in large-scale initiatives, including leadership support,36 this study was designed to test whether adherence to the Project JOINTS QI methods and participation (or ‘engagement’) in Project JOINTS activities could facilitate the adoption of the SSI prevention bundle in hospitals and consequently reduce SSI rates after hip and knee arthroplasty.

Study design

Study hypotheses

Because data on SSI rates (outcomes) at the hospital level are not available to us at this time, we focused on the reported uptake of the SSI bundle as the primary measure of effectiveness. We tested two hypotheses based on the QI literature and prior IHI experience. Previous research shows that adherence to the QI methods (such as the use of Plan-Do-Study-Act cycles) helps hospitals implement QI initiatives.37 Similarly, prior IHI experience with large-scale QI initiatives suggests that adherence to QI methods, such as the establishment of a QI plan with clear aims, reliance on multidisciplinary implementation teams and the conduct of small-scale tests of change, is likely to positively affect implementation of a QI initiative.6 Therefore, we hypothesised that hospitals that adhere to the Project JOINTS QI methods will be more likely to adopt the enhanced SSI prevention bundle than hospitals that do not.

Furthermore, results of a recent literature review on determinants of success of QI collaboratives18 and previous IHI experience38 suggest that hospital engagement in QI activities may positively impact intervention implementation. Because the IHI campaign infrastructure is designed to help hospitals learn about the evidence supporting intervention components and QI methods, as well as benefit from the experience of IHI faculty and their peers,22 we hypothesised that the level of hospital engagement in Project JOINTS (ie, use of campaign materials and tools and participation in interactive activities) will be positively associated with bundle adoption.

Data collection

We tested these hypotheses using self-reported, cross-sectional interview data collected in five states that participated in Project JOINTS. We developed a structured interview protocol to collect self-reported data on hospital adherence to the Project JOINTS QI methods, hospital engagement in Project JOINTS and uptake of the five-component enhanced SSI prevention bundle. Adherence to Project JOINTS QI methods referred to the development of an implementation plan, the deployment of an implementation team and the use of small-scale tests of change, such as conducting a pilot study or implementing the project for a selected group of patients first. To measure hospital engagement in Project JOINTS, we asked participants to indicate which of the 10 Project JOINTS tools and activities their hospital used or participated in (see figure 2 for a complete list of campaign activities, materials and tools). To measure bundle adoption, we asked the following question: ‘Are staff at your hospital now doing all five of the components?’

Figure 2

Hospital use of Project JOINTS (Joining Organizations IN Tackling SSIs) resources (N=73).

We also asked questions about implementation of each of the five individual bundle components. We asked whether hospital staff had (a) performed the bundle component consistently prior to Project JOINTS; (b) initiated the component as a result of participation in Project JOINTS; (c) performed the component inconsistently before Project JOINTS, but now were performing it consistently; or (d) other (please specify). We also asked about barriers and facilitators to bundle adoption. The interview protocol included both close-ended and open-ended questions, and we encouraged participants to elaborate on their answers. The research protocol was reviewed and approved by our Institutional Review Board.

The interviews with representatives of hospitals enrolled in Project JOINTS were conducted by telephone between November 2012 and January 2013. Based on the information provided by IHI, we contacted all eligible hospitals that had enrolled in Project JOINTS in the five intervention states (n=109 hospitals). Our recruitment procedures including a first contact by email sent to the identified Project JOINTS contact at each participating hospital. We followed up non-respondents with a second email and then a telephone call. Each interview lasted for approximately 45 min; participant responses were entered into SelectSurvey, a web survey tool.

Quantitative data analysis

To conduct the quantitative analyses, we constructed several indices based on the interview items noted above. We created an index of adherence to the Project JOINTS QI methods by adding the number of three QI methods (range 0–3) a respondent reported that his/her hospital used within Project JOINTS. We created a hospital engagement index by summing the number of Project JOINTS tools and activities each hospital used (range 0–10). We also measured the type of resources hospitals used by creating two new indices: number of interactive activities (range 0–4) and number of printed and online Project JOINTS materials and tools (range 0–6). Finally, we created a dichotomous (yes/no) measure of complete bundle adoption, which we used as a primary measure of intervention uptake. We also created a variable measuring the number of newly initiated (response category b) or improved (response category c) bundle components, which was used as a secondary measure of intervention uptake.

Covariates included the following hospital characteristic variables: hospital size (small, 2–42 beds; medium, 43–115 beds and large, 116–381 beds; calculated based on the number of beds tertiles obtained from 2011 American Hospital Association (AHA) data); location (urban vs rural, as defined by participants, and state); volume of hip and knee arthroplasty (above or equal/below 250 annual procedures (median value) using the 2011 Hospital Compare Medicare data); and self-reported previous experience with IHI campaigns, other SSI prevention bundles and organised patient safety initiativesi (yes/no to each).

To test our hypotheses, depending on the outcome measures, we used logistic or Poisson regression with robust SEs to account for potential heteroscedasticity and tested for mediation effects. We first modelled the relationship between adherence to QI methods and two measures of intervention uptake—complete bundle adoption and initiation of and increased adherence to bundle components. Second, we modelled the relationship between hospital engagement and measures of bundle uptake. Finally, we modelled the impact of both predictors together to explore mediation effects.

Qualitative data analysis

To identify and summarise themes from responses to the open-ended questions, we developed a hierarchically organised codebook based on the interview protocol. We incorporated themes from the QI literature and previous IHI campaigns, and new unanticipated themes that emerged from the interviews. Responses to open-ended questions (such as the question about barriers to complete bundle adoption) were reviewed and systematically coded (using MAXQDA 10 qualitative data management software) into a series of 8–10 response categories based on the codebook developed for each question. Two qualitative researchers coded the data independently and then reviewed coded text to ensure coding consistency; disagreements were discussed until consensus was reached.39 ,40

Results

Our final sample consisted of representatives of 73 of the 109 hospitals participating in Project JOINTS (response rate of 67%). Response rates varied across the five states—from a low of 60% of eligible hospitals to a high of 83%. Each of our respondents was significantly involved in Project JOINTS and knew about the project-related activities in his/her hospital. The majority of respondents were infection control practitioners, followed by directors/managers of surgical services and directors of clinical quality. Respondents had a relatively long tenure at their hospital (mean=15.3 years; SD=11.85; range 5 months–43 years) and had spent, on average, 7.1 years (SD=7.11; range 3 months–30 years) in the current role.

Most of the hospitals in our sample were small (41%) or medium-sized (36%) (see table 1). About an equal number were located in rural and urban settings and approximately equal numbers had high and low arthroplasty volume. Most hospitals reported participating in prior IHI campaigns (59%), SSI prevention activities (63%) and organised patient safety initiatives (90%). More than one-third (36%) of the hospitals in our sample was located in a single state.

Table 1

Sample descriptions (N=73)

Use of QI methods and engagement in project JOINTS

More than two-thirds of responding hospitals (n=50) used at least one of the Project JOINTS QI methods. Out of the 50, 90% had an implementation team, 52% developed an implementation plan and 36% conducted a small test of change (see table 1).

Participating hospitals were variably engaged in Project JOINTS. On average, hospital QI leaders reported using slightly more than half of all available Project JOINTS resources (5.4 out of 10). They reported participating in 2.45 (out of four) interactive activities (SD=1.09) and using 2.95 (out of six) online or print Project JOINTS materials and tools (SD=1.99). Webinar calls and electronic communications were the most popular interactive activities with more than four-fifths of hospital QI leaders reporting that their hospitals had participated in them (figure 2). Among the project materials and tools, the ‘How to Guide’, the ‘improvement tools’ and the surgeon evidence handouts were the most popular: roughly three-fifths of respondents reporting using each.

Adoption of the enhanced SSI prevention bundle and its components

The majority of respondents (55%) reported that their hospital adopted the complete (five-component) enhanced SSI prevention bundle (table 1). Figure 3 suggests, however, that these hospitals were already familiar with many of the bundle components: on average, they reported using three out of five enhanced SSI prevention bundle components prior to Project JOINTS (appropriate hair removal, appropriate use of prophylactic antibiotic and the use of alcohol-containing antiseptic were the most commonly reported). Approximately one-third of the hospitals reported not initiating or increasing adherence to any of the bundle components during Project JOINTS (see online supplementary appendix B). Hospitals that did initiate or increase adherence tended to focus on two of the three new components suggested by Project JOINTS—the 3-day use of CHG soap (50%) and screening for SA carriage and decolonisation of carriers (40%) (figure 3).

Figure 3

SSI prevention bundle component adoption (N=73). SSI, surgical site infection; CHG, chlorhexidine gluconate.

Does adherence to the QI methods affect bundle adoption?

Reported adherence to the QI methods was positively associated with complete bundle adoption (r=0.33, p≤0.001) and initiation of or improved adherence to bundle components (r=0.51, p≤0.001). Controlling for hospital characteristics and previous implementation of SSI bundle components, for each additional QI method a hospital implemented, the odds of adopting the complete enhanced SSI prevention bundle increased; this result was statistically significant (OR=2.53, 95% CI 1.18 to 5.42, p≤0.05; table 2, model A). However, adherence to the QI methods was not significantly associated with initiation or improved adherence to individual bundle components (incidence rate ratio (IRR)=1.16, p>0.1; table 3, model A).

Table 2

Logistic regression models predicting complete bundle adoption (N=73)

Table 3

Poisson regression models predicting initiation/improved adherence to bundle components (N=73)

Does the level of hospital engagement in Project JOINTS affect bundle uptake?

The level of hospital engagement with Project JOINTS was positively and statistically significantly associated with adherence to Project JOINTS QI methods (r=0.52, p≤0.01), complete bundle adoption (r=0.23, p≤0.05) and initiation of or improved adherence to bundle components (r=0.5, p≤0.001). Controlling for hospital characteristics and previously implemented SSI prevention bundle components, more engaged hospitals had better odds of adopting the entire bundle (OR=1.45, 95% CI 1.09 to 1.93, p≤0.05; table 2, model B). The odds of complete bundle adoption were 1.45 times higher for each additional Project JOINTS tool or activity a hospital reported using. The association of hospital engagement with the number of bundle components initiated or newly adhered to was statistically significant but small (IRR=1.12, 95% CI 1.03 to 1.22, p≤0.01; table 3, model B). Each additional hospital activity or tool used was associated with a 12% increase in the number of bundle components a hospital QI leader reported initiating or improving adherence to.

Which is more important: adherence to the QI methods or hospital engagement?

When we entered both predictors into the multiple regression model, adherence to Project JOINTS QI methods (OR=2.05, 95% CI 0.91 to 4.62, p≤0.1; table 2, model C) and level of hospital engagement (OR=1.32, 95% CI 0.97 to 1.81, p≤0.1; table 2, model C) both had a positive effect on complete bundle adoption. However, only hospital engagement (IRR=1.11, 95% CI 1.02 to 1.21, p≤0.05; table 3, model C), and not adherence to Project JOINTS QI methods (IRR=1.04, 95% CI 0.87 to 1.26, p>0.1; table 3, model C), was associated with initiation of or improved adherence to individual bundle components.

To determine whether the type of engagement in Project JOINTS is associated with bundle adoption, we differentiated hospital engagement into participation in interactive campaign activities and the use of printed and online campaign materials and tools. We found that complete bundle adoption was only associated with adherence to Project JOINTS QI methods (OR=2.01, 95% CI 0.88 to 4.62, p≤0.1; table 2, model D), suggesting that the level rather than type of engagement, along with the use of QI methods, had stronger positive associations with the adoption of all bundle components. We also found that initiation of or improved adherence to bundle components was statistically significantly associated only with the use of printed and online materials and tools (IRR=1.13, 95% CI 1.01 to 1.27, p≤0.05; table 3, model D).

What are the barriers to complete bundle adoption?

Hospital representatives we interviewed cited several barriers to adopting all five bundle components. One-third of all respondents in hospitals that had not implemented the complete bundle reported lack of physician buy-in and staff resistance as the main barriers. Explained one interviewee, Our facility is owned half by physicians, and they don't belong to the same organization. They are not our employees. So if they decide not to do something, they don't. We always try to compromise and have tried to come up with a process that will make them happy and that will serve the patients as well, but frankly, we have failed. We failed at getting Project JOINTS on board here.

The second and third most commonly cited barriers to complete bundle adoption were the cost of antibiotics and CHG soap (18%) and the logistics of screening and bathing (18%). Indeed, several individuals noted that the logistics of implementing some bundle components (eg, screening and bathing) were more difficult to implement than others (eg, hair removal). Logistical challenges included coordination among staff, calls to insurance companies, increased treatments and dependence on patients’ actions outside of a care facility.

Finally, 15% cited a perceived lack of evidence supporting some bundle components. As described by one interviewee, ‘I do not see any literature that says that three days are necessary’. ‘All the literature I've looked at says that [bathing/showering one day before surgery] is enough. It's already hard to get this going 100% of the time,’ said another. Similarly, another 15% reported that their hospital had plans to start using the bundle components soon, but wanted to see the results of this evaluation study. Interestingly, 9% of hospital respondents did not feel the need to adopt the bundle because SSI was ‘not a problem in the hospital’.

Discussion

Our results supported both of our study hypotheses. Both adherence to the QI methods and the level of hospital engagement in Project JOINTS were positively associated with the adoption of the complete SSI prevention bundle. Controlling for hospital characteristics and previously implemented bundle components, we found that adherence to more of the QI methods was associated with higher odds of adopting the complete bundle. This association between adoption of evidence-based practices and QI methods adherence (developing a plan, forming an implementation team and conducting small-scale tests of change) is consistent with prior literature suggesting that basing a campaign on systematic use of QI methods is an important pathway towards its effectiveness.6 ,37

In addition, we found that among hospitals participating in Project JOINTS, those that were more engaged in Project JOINTS were more likely to implement the complete bundle and initiated or improved adherence to a larger number of the individual bundle components. It is possible that having access to experienced faculty members and carefully developed project materials, as well as learning from peers during in-person and online interactive activities, helped hospitals address barriers to adopting new evidence-based practices in their own settings. Hospital engagement was also an important factor that facilitated intervention implementation in the IHI Breakthrough Series collaboratives.38 This result may suggest that large-scale QI campaigns can be more successful if they focus additional efforts on encouraging participating hospitals to engage in more campaign activities and use campaign materials; nonetheless, we cannot exclude the possibility that more engaged hospitals are simply more receptive to QI.

All else being equal, hospitals that used more Project JOINTS printed and online materials and tools were more likely to initiate or improve adherence to individual bundle components compared with hospitals that used fewer Project JOINTS resources. This finding may be somewhat surprising given prior research suggesting that printed educational materials have limited effects on professional practice outcomes.41 ,42 We note, however, that Project JOINTS did not merely distribute copies of clinical guidelines. Instead, IHI staff created tailored Project JOINTS materials for specific uses and audiences (eg, surgeon, family and patient handouts) to facilitate their buy-in and acceptance of Project JOINTS. Tailoring intervention materials and ensuring that they are easy-to-use and readily accessible to campaign participants (eg, via project website) may be very important. Creation of high-quality customised intervention materials that clearly describe the evidence base behind intervention components may facilitate adoption of best practices by specific groups of stakeholders (in this instance, orthopaedic surgeons).

More than half of our respondents reported that their hospitals had adopted the complete bundle. Among those hospitals that did not adopt the complete bundle, 36% did not improve or increase adherence to any bundle components as part of Project JOINTS (data not shown). Lack of surgeon buy-in and staff support for implementing the bundle are well-known implementation challenges in the use of QI collaboratives43 and were the two most cited barriers in this study.

Moreover, Project JOINTS hospitals often struggled to implement the two components (using CHG soap for at least three days before the surgery and preoperative nasal screening and SA decolonisation) that they reported to be the most logistically challenging and for which there was still some scepticism about the evidence supporting their effectiveness. The finding about logistical challenges illustrates the difficulty of changing processes that require substantial coordination of multiple services and departments, thereby supporting an argument that the effects of a QI intervention ‘may be mediated by the nature of the processes being targeted and the amount of control participants have over them’.13 Hospital representatives also noted that in general they conduct their own evaluation of the evidence supporting each bundle component and implement components only when they are persuaded that the evidence of effectiveness meets their own threshold. This observation about claims of inadequate evidence is consistent with the results of previous research.12 ,44 Several respondents hoped Project JOINTS would produce the evidence necessary to justify complete bundle adoption.

This study has several limitations. First, it relies on self-reported, cross-sectional interview data collected from a single representative of each participating hospital. Senior personnel participating in our study might have been misinformed about the implementation of campaign activities. Although frequently used to evaluate the process and outcomes of QI collaboratives, self-reported data may also be subject to recall bias. To minimise recall bias, we started collecting these data within a month after the Project JOINTS programme was completed. If additional changes in bundle adoption took longer to occur, our findings may underestimate the effect of the intervention.

Second, responses may have been affected by social desirability bias. Senior personnel participating in a QI campaign may exaggerate the extent of bundle adoption. While we cannot quantify the magnitude of this bias, we note that several participants reported both positive and negative feedback about their Project JOINTS experiences (36% reported that they did not initiate or improve adherence to any components of the SSI prevention bundle). Nevertheless, it is possible that respondents who report higher adherence to QI methods and engagement in Project JOINTS may also be more likely to report bundle uptake because they may believe that if their hospital adhered to QI methods or engaged in Project JOINTS, it must also have improved adherence to bundle components (and vice versa). According to cognitive dissonance theory, individuals commonly search for consistency in their beliefs and perceptions; they try to avoid inconsistencies that may lead to discrepancies between their beliefs and behaviours by changing their beliefs and perceptions to reduce or eliminate the dissonance.45

Finally, while we controlled for a number of hospital characteristics that describe intervention context, our interview protocol did not include questions on several contextual variables, such as organisational culture, teamwork and leadership support; these may have affected the implementation of QI. Although this was a large QI campaign, this sample of voluntary participants tended to be smaller and more often rural, which may limit generalisability of these results to other types of hospitals.

Looking inside the ‘black box’ of a QI campaign, we find that—from the point of view of participants—certain campaign strategies were more helpful than others in assisting hospitals to reach their SSI prevention goals. Our findings suggest four opportunities to enhance the effectiveness of large-scale QI campaigns.

First, actively engaging hospitals by encouraging them to use campaign resources, such as online and printed materials/tools and interactive activities, may help hospitals to initiate or improve adherence to new bundle components. Customising materials so that they are tailored to the needs of participating hospitals and their stakeholders can help hospitals address barriers to adopting new QI practices in their own setting.

Second, printed and online campaign materials that are self-explanatory offer a forthright description of the latest clinical evidence and explain the advantages and potential benefits of changing current clinical practice appear to be more effective. For surgical QI campaigns, particular emphasis should be placed on the materials distributed directly to surgeons; materials tailored specifically for this audience could be crucial for engaging them in the campaign and getting their support for clinical practice change. Lack of surgeon buy-in was one of the two most cited barriers to implementing the Project JOINTS bundle.

Third, hospital engagement in campaign activities may help participants learn and share experiences with one another. We found that hospital engagement was positively associated with both adoption of the complete bundle and with initiation and adoption of individual components. Local Rapid Spread Network organisations may be better positioned to engage hospitals in QI activities.

Finally, campaigns that promote adherence to specified QI methods may be more likely to achieve bundle adoption. Experienced campaign faculty can help hospital implementation teams develop implementation plans and advise on how to conduct small tests of change. Developing a QI plan and conducting small tests of change may have been underused in this campaign.

In conclusion, our findings provide some support for the use of QI methods and promotion of active hospital engagement in a QI campaign as a way to increase adoption of evidence-based, multicomponent patient safety bundles in orthopaedic practice.

References

View Abstract
  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Contributors DK and MSR led the study design. DK supervised the team that collected the study data and performed the qualitative analyses. CH helped collect the quantitative data and carried out the statistical analyses. All authors participated in the interpretation of the results and drafting and critical revision of the manuscript. ECS was the principal investigator of the study.

  • Funding This study was funded by a grant (R18 AE000051) from the US Department of Health and Human Services.

  • Competing interests None.

  • Ethics approval This study was reviewed and approved by the Human Subjects’ Protection Committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • i This variable had no significant bivariate associations with the outcome variables and was not included in the final models.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.