You are here

Article Text

Article menu
Download PDFPDF
Original research
A qualitative study of the variable effects of audit and feedback in the ICU
  1. Tasnim Sinuff1,2,
  2. John Muscedere3,
  3. Linda Rozmovits4,
  4. Craig M Dale5,
  5. Damon C Scales1,2
  1. 1Department of Critical Care Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada
  2. 2Interdepartmental Division of Critical Care, University of Toronto, Toronto, Ontario, Canada
  3. 3Department of Medicine and Critical Care Program, Queens University, Kingston General Hospital, Kingston, Ontario, Canada
  4. 4Independent Qualitative Researcher, Toronto, Ontario, Canada
  5. 5Lawrence S. Bloomberg Faculty of Nursing, University of Toronto, Toronto, Ontario, Canada
  1. Correspondence to Dr Tasnim Sinuff, Department of Critical Care Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada M4N 3M5; taz.sinuff{at}sunnybrook.ca

Abstract

Background Audit and feedback is integral to performance improvement and behaviour change in the intensive care unit (ICU). However, there remain large gaps in our understanding of the social experience of audit and feedback and the mechanisms whereby it can be optimised as a quality improvement strategy in the ICU setting.

Methods We conducted a modified grounded theory qualitative study. Seventy-two clinicians from five academic and five community ICUs in Ontario, Canada, were interviewed. Team members reviewed interview transcripts independently. Data analysis used constant comparative methods.

Results Clinicians interviewed experienced audit and feedback as fragmented and variable in its effectiveness. Moreover, clinicians felt disconnected from the process. The audit process was perceived as being insufficiently transparent. Feedback was often untimely, incomplete and not actionable. Specific groups such as respiratory therapists and night-shift clinicians felt marginalised. Suggestions for improvement included improving information sharing about the rationale for change and the audit process, tools and metrics; implementing peer-to-peer quality discussions to avoid a top-down approach (eg, incorporating feedback into discussions at daily rounds); providing effective feedback which contains specific, transparent and actionable information; delivering timely feedback (ie, balancing feedback proximate to events with trends over time) and increasing engagement by senior management.

Conclusions ICU clinicians experience audit and feedback as fragmented communication with feedback being especially problematic. Attention to improving communication, integration of the process into daily clinical activities and making feedback timely, specific and actionable may increase the effectiveness of audit and feedback to affect desired change.

  • Audit and feedback
  • Implementation science
  • Qualitative research
  • Quality improvement

https://doi.org/10.1136/bmjqs-2015-003978

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Audit and feedback has been proposed as an important approach for improving the quality of intensive care unit (ICU) services worldwide.1–3 As a means of attenuating practice variation contributing to poor clinical outcomes,4 ,5 audit and feedback has become an integral component of continuous quality improvement and patient safety initiatives.6 ,7 Outside the ICU setting, when included in multifaceted improvement strategies, audit and feedback is theorised to motivate providers to improve care, particularly when practice is identified as inconsistent with peers, institutions or accepted guidelines.8–10 However, in the ICU, this strategy is often linked to research-informed initiatives known to reduce patient mortality, duration of mechanical ventilation and ICU length of stay.11 ,12 In response to these advantages, considerable managerial commitments have been made to audit and feedback in ICUs internationally.6 ,13–24

    Notwithstanding its potential benefit, a large gap exists in our understanding of why audit and feedback may or may not work in the ICU context.3 ,12 ,25 This uncertainty emerges, in part, from the current body of evidence outside the ICU, which describes wide variation in the effectiveness of audit and feedback.8–10 Analysis of systematic review data on audit and feedback26 using theoretical constructs of Feedback Intervention Theory27 and Control Theory28 suggests that the existing literature lacks a theory-driven approach to the development of audit and feedback interventions. Using theory to develop feedback interventions, it is proposed, may improve its effectiveness and thereby improve quality of care.29

    Variable effectiveness within the ICU11 may be traced to published ICU reports whereby audit and feedback is undertaken in different ways: at the point-of-patient care in two,23 ,24 weekly in two,15 ,18 monthly in four,14 ,16–17 ,22 at a single time in one19 and not specified in two20 ,21 studies. Across reports, feedback is delivered through various channels including electronic, print and verbal processes. Moreover, feedback alternates between individual and group performance. Because these audit and feedback methods are complex and diverse, they may be difficult to operationalise. Uncertainties regarding the effectiveness and optimal approaches to audit and feedback suggest that further study is warranted.

    Quality improvement in the ICU has been described as arduous and often emotionally laden work for ICU stakeholders.5 ,6 To understand why audit and feedback may or may not work in complex environments such as the ICU, we need to understand the relevancies and problems stakeholders identify in the processes that comprise this improvement practice. Hence, our objective was to understand both the experiences of ICU clinicians and leaders with audit and feedback across a range of community and academic ICUs and the core mechanisms that lead to the success or failure of the implementation of audit and feedback. We therefore conducted a qualitative study30 ,31 to understand factors that influence the effectiveness of audit and feedback in the ICU.

    Methods

    We wanted to understand the individual and organisational barriers to implementing audit and feedback and preferences for the types of audit and feedback interventions to support behaviour change from the perspectives of ICU clinicians and leaders. Although there are theoretical frameworks to explore the issues of implementation of strategies such as audit and feedback at both the individual and organisational levels,32–34 we approached this area without a pre-existing theoretical construct as audit and feedback is at its relative infancy in the ICU setting. Hence, we used a qualitative descriptive approach35 ,36 incorporating techniques from grounded theory.31

    Setting, participants and sampling

    We conducted this study in a representative sample of five academic and five community adult ICUs in Ontario, Canada. Participating units had between 10 and 20 ICU beds and all ICUs were managed under an intensivist or closed model of care.37 We included ICUs, which used audit and feedback variably across sites and across different best practices. The frequency of audit and feedback practices ranged from infrequently (less than quarterly) to routinely integrated into their culture (more than quarterly for at least 12 months). We included both academic and community hospitals as our prior work in this area suggested that different types of institutions have variable opportunities and experiences with audit and feedback.5

    We sought participants with a variety of professional roles and insights. Hence, participants from each site included attending physicians, nurses, respiratory therapists, pharmacists and ICU administrators (physician directors, nurse and respiratory therapist managers). One academic centre served as the pilot site for interviews.

    Initially, typical case sampling was used as a purposive strategy to recruit participants who represented common members of multidisciplinary ICU team including clinicians and administrators involved in the day-to-day management of critically ill, mechanically ventilated patients.30 ,31 Participants were recruited until no new information was forthcoming (saturation).31

    Data collection

    We used semistructured, in-person in-depth interviews for the majority of participants. For logistical reasons, 12 interviews with staff at two of the facilities were conducted by telephone. We developed our interview guide based on the audit and feedback literature8 ,10 and our study objective. Preliminary interview guides were developed for each clinician group (physician, nurse, respiratory therapist) and were piloted by conducting two interviews per ICU clinician group (intensivists, nurses, respiratory therapists), and with the nurse manager and physician director from the pilot site. The interview guide was revised following a preliminary analysis and team meetings (see online supplementary appendix).

    The final number of participants per ICU clinician group was unequal across clinician groups and hospitals as the targeted sampling method was sensitive to the relative number of clinicians and their viewpoints at each participating ICU. All interviews were digitally audio recorded for verbatim transcription and were an average of 35 min in length.

    Analysis

    Analyses were conducted using constant comparative methods.31 Two investigators (TS, LR) developed the initial coding framework and subsequent necessary revisions. Initial coding was conducted by one of the investigators (LR) and then in duplicate by another investigator (TS) for a selection of codes considered more interpretive to ensure dependability of coding.38 Constant comparative methods included searches for disconfirming evidence. The codes and emergent themes were discussed in detail by two of the investigators (TS, LR). Subsequently, all emergent categories and themes were analysed (TS) to understand relationships between the codified phenomenon and the circumstances contributing to the core problem.31 Data were managed using HyperResearch software (Randolph, Massachusetts, USA).

    Ethics

    Approval to conduct this study was obtained from all 10 participating institutional research ethics boards. Written informed consent was obtained from each participant.

    Results

    We conducted 72 interviews, including 17 intensivists, 34 nurses (bedside and managerial/education roles), 17 respiratory therapists (bedside and managerial/education roles), one dietician and three pharmacists (table 1). Participants’ critical care experience varied and included novice and veteran clinical workers (table 2).

    View this table:
    Table 1

    Number of participants by participating hospital

    View this table:
    Table 2

    Participants by number of years in critical care

    Eight ICUs routinely used audit and feedback. In two ICUs audit and feedback was infrequently used; in one, this was due to resource limitations and in the other it was a matter of the prevailing culture, which favoured mandatory approaches for implementation of best practices (eg, preprinted order sets, restriction of clinicians’ options, etc). Audit and feedback was used primarily for guidelines or protocols used to reduce central line infections and ventilator-associated pneumonia.

    Discontinuous communication

    Audit and feedback might have inconsistent or suboptimal effects in ICU due to the impaired nature of communication of the entire process. As identified by participants from the ICUs that conducted audit and feedback routinely, the core problem was that clinicians experienced audit and feedback as fragmented or discontinuous communication, because of the manner in which both processes were communicated to bedside clinicians (tacit, explicit, verbally and non-verbally implemented, enacted and perceived). The communication was fragmented and variable in its effectiveness.

    Clinicians felt that audit, as it was generally conducted within their unit, lacked transparency and credibility, and that feedback was not timely, complete or actionable. Participants conveyed that the results of this discontinuous and variable nature of communication of the entire process of audit and feedback had negative ramifications—including perceptions of blame and feelings of marginalisation. Many felt disconnected from the entire process.

    Audit process: lack of transparency, credibility and disenfranchisement

    Understanding of audit processes was often limited to those responsible for conducting the audits and to a few individuals with direct involvement in quality improvement. For most participants, auditing was an opaque process carried out by unknown individuals for unspecified reasons. The lack of transparency often resulted in alienation of staff from quality improvement efforts and lost opportunities for learning. There was a sense that bedside ICU staff felt disenfranchised: their opinions about the process were not sought nor were they informed of the process. I'm aware there's people in the unit collecting data but I don't know if it's for audit and feedback or a research project … unless they need your authorization to do something they usually don't communicate with us directly. [Physician] [Participant eight] The forms just started showing up … nobody's really explained, “We're doing it for this purpose,” … the management or the educators are saying, “Well, why didn't you fill it out? Why didn't you fill it out?”…That goes to the bottom of my list because I don't know what it's about and I don't know what it's for. [Bedside Nurse] [Participant 25] We're missing an opportunity to actually improve the process because the auditors come and run away … if all these initiatives are really about helping save lives … then do the audit, get the objective measurement … mark it down as you see it and then help us change. [Physician Director] [Participant 55]

    Clinicians also conveyed perceptions of surveillance and blame. For many and especially bedside nurses, the audit process held negative associations because they lacked information sharing. Clinicians often felt that they were under surveillance.

    Others felt marginalised. Many targeted care practices were perceived to be nursing focused and information sharing about the audit process often did not reach colleagues outside of that professional group. Night-shift clinicians also felt overlooked. They expressed dissatisfaction at having to rely on impersonal forms of communication such as e-mail or posted notices rather than in-person communication or direct feedback about important best practice initiatives. It doesn't make you feel very good when you think your Big Brother's watching … I like to know the reasons why we're doing something and like to think that it's improving things. [Bedside Nurse] [Participant 29] They would go through my stuff and not tell me what they were doing …It's kind of sneaky … it feels like I'm not part of the team … [Bedside Nurse] [Participant 53] We're not really included, we're the other healthcare professional, we're not physicians, we're not nursing, we're in the other category, so they just kind of whoosh…bypass us … [Respiratory Therapist] [Participant 50] You never see the manager, you never see the educator, you don't see anybody and if the information about these best practices aren't clearly passed along they get lost on nights … we're left to rely on e-mail or stuff posted on the bathroom wall and that's about it … [Bedside Nurse] [Participant 25]

    Feedback: need for more timely and actionable feedback

    Feedback was clearly perceived to be an important tool for change, but was perceived to be more difficult to implement effectively than audit due to limited resources, logistical challenges, poor communication and change fatigue. Feedback data were rarely provided to front-line care providers in a timely, complete or actionable manner that could support behaviour change. When it was provided, there was a reliance on paper and electronic texts as opposed to direct communication. Feedback was often circulated by e-mail, posted notices or through staff meetings, none of which were felt to be particularly effective. Participants expressed a strong preference for direct face-to-face communication that would allow for education, discussion and feedback connected to specific patients. Many participants pointed out that while audit data might be fedback, this did not necessarily make for effective feedback, which they conceived as being timely, transparent and actionable information.

    The lack of timeliness and specificity was of primary concern to clinicians. Overall, there was a strong sense of a disjointed process and incomplete communication as it relates to an active work process. Feedback was often perceived to be irrelevant because of delayed reporting. However, some participants cautioned against interpreting short-term data in the absence of longer term trends since short timescales also meant small denominators. Timeliness, in other words, was not just about the speed with which feedback was delivered but, more importantly, about a meaningful sense of temporality. To achieve this, both the short-term picture and the longer term trends needed to be addressed. How useful is it for me to know that in the month of April we had two line infections. Nobody knows whose patient that was, we don't know how it was breached and we don't know what happened to that patient [Physician] [Participant 40] If you feed back data to people quickly it's always on a small number … it's really hard for us to know what “your patients” means anyway, you'd be talking about feeding back data on something like twenty patients, many of whom the exposure wasn't just to me. [Physician] [Participant ten] Just having one month in isolation is not necessarily the most useful sort of thing … things go up, things go down – it's more useful to see that you've got a trend going on over time … [Pharmacist] [Participant 35]

    In order to be useful, participants felt that feedback needed to be concrete, patient oriented and actionable. Feedback was difficult for many staff members to engage with because it was too abstract, passive and disconnected from care at the bedside. Bedside clinicians who were not research oriented, in particular, could not relate to the volumes of data presented. Overall, bedside clinicians indicated that feedback lacked directives for improvement and specific, clinically focused actions, which can be acted upon more immediately. If you find out about a problem after months after the fact it's not as useful to you as opposed to somebody coming to me right now and saying we have a violation of the VAP bundle … and then we say, “Okay, let's correct it.” [Physician] [Participant 40] So the feedback, positive or negative is more what I focus on, rather than the actual numbers … The actual numbers I have no idea … it takes a back seat for me because I'm not focused on that, I'm focused on what am I doing for these people. [Bedside Nurse and Charge Nurse] [Participant 62]

    Strategies to improve the effectiveness of audit and feedback

    To improve the effectiveness of audit and feedback, clinicians suggested improving information sharing, encouraging peer-to-peer discussions about quality improvement and engaging leadership. ICU clinicians felt that improved information sharing about all aspects of audit and feedback would address staff concerns about focusing on seemingly arbitrary or unjust measures and encourage greater sense of ownership of the results. Just to understand what it is you're going to be measured by, it'll help you to…personalize it, “I really do have a responsibility to do X because it will make a difference” [Patient Care Manager] [Participant 39] If it was more known…what they were actually going to be doing with the information it might be easier…to comply. [Bedside Nurse] [Participant 23]

    Participants conveyed that a peer-to-peer discussions regarding quality improvement in the ICU would generate greater clinician engagement about quality improvement initiatives in general. A more peer-driven collaborative process would be better accepted and would more effectively achieve sustainable behaviour change through improved communication, greater receptiveness to education and increased compliance. Several ICUs also found that incorporating feedback into the daily rounds conversation resulted in greater consistency. This was seen to foster greater ownership for best practice implementation since these could be directly linked to patients. If you hear information…from your peer you're more likely to accept it … it [becomes] a conversation. [Bedside Nurse] [Participant 17] It leverages against people's sense of duty and ownership of their patients … it's very powerful in implementing change. [Physician Director] [Participant 55]

    Engagement of leadership on levels, from nursing and physician to senior administration, was perceived to be a very important aspect of successful audit and feedback. In particular, the leadership role of physicians was a prominent factor for many. Although the ICU team was multidisciplinary, without physicians’ active and sustained engagement, change was difficult if not impossible. Although the flattened hierarchy within healthcare is advocated, it is not yet established within the work environment. Participants wished for nurses to embrace the opportunity for leadership. The capacity of nurses to implement change depended on the consistency of physician response. Being well supported and having recognition of achievements by hospital senior management were seen as imperative to increase clinicians’ confidence in, and willingness to engage with the audit and feedback process. There's still a widely-held view that the physician is the leader of the Critical Care team … they really drive it. [Nurse Educator] [Participant 17] Most of the things that we're trying to implement are nursing best practices … what physicians need to do is to create an environment … that allows nurses to … take leadership on those. [Physician] [Participant ten] There has to be … the institutional will to make this happen and to be supported by very senior people… At the corporate level it needs to be supported, championed and rewarded. [Physician Director] [Participant 45]

    Discussion

    We conducted a qualitative study of multidisciplinary ICU team members’ perspectives of audit and feedback in the ICU and found that the overall perception was that audit and feedback was potentially useful but often failed to affect clinical care because of the way that it was implemented. Audit and feedback was frequently experienced at the individual and professional level as discontinuous communication. The audit process was felt to lack transparency and credibility, and feedback was felt to be untimely, incomplete and often not actionable. The impaired nature of communication and its negative ramifications may be a significant mechanism underlying the suboptimal effect of audit and feedback in the ICU.

    Several suggestions emerged about how to improve the process of audit and feedback in the ICU. These included improved information sharing about the audit and feedback process and rationale for change; implementation of peer-to-peer discussions about quality improvement to avoid a top-down approach and more positive engagement in audit and feedback by senior management.

    In our study, participants strongly expressed the desire for timely and contextualised feedback. This is consistent with both Feedback Intervention Theory39 and existing empirical research outside the ICU setting. Feedback Intervention Theory39 posits that in order to change behaviour, feedback should be timely, actionable and provide data required to change behaviour. Empirical studies corroborate these tenets. For example, audit and feedback appears to be most effective for increasing adherence to clinical practice guidelines when feedback is timely,40 individualised41 ,42 and non-punitive.43–45 Feedback cues should follow the following sequence (from greatest to least barriers of feedback effectiveness): timely—individualised—non-punitive, which makes feedback increasingly actionable.29 Hysong et al29 posit a new sequential model, which adds customisability to these aforementioned three feedback intervention cues, to enhance feedback.

    Our study has important implications for conducting audit and feedback in the ICU setting. First, how the audit process is communicated and how feedback is delivered to ICU clinicians play a significant role in the acceptance of the process and the results. Our results are consistent with Goal Setting Theory46 ,47 (feedback moderates individual motivation; goals need to be specific and feedback enhances performance) and Control Theory48 (behaviour is goal driven and feedback directs divergent behaviour back towards the desired performance goal). Clinicians in our study consistently felt that feedback must be specific, with specific and attainable outputs to be achieved. Accordingly, feedback may be enhanced by setting performance targets and implementing specific action plans.28 Second, from a practice perspective, whether for administrative or developmental purposes, audit and feedback is more than a simple reporting of performance data. Feedback needs to be both meaningful and tailored to the needs of the team members in order for clinicians to improve their performance.42 ,43 ,49 Some of the mixed findings in the ICU audit and feedback literature could be partially explained by differences in the audit and feedback characteristics.5–15 Future research should consider the limitations of prior ICU studies when designing audit and feedback interventions. Moreover, the aforementioned theories may be used to design appropriate feedback interventions to improve adherence to best practices and improve quality of care.28 ,29

    This study is limited in that we interviewed clinicians once, which may not reflect their perspectives longitudinally nor describe the material circumstances of their work. However, since it was not our intention to conduct an ethnography of this area, our approach was an appropriate qualitative methodology to understand the social effects of audit and feedback. This method is helpful in identifying core problems and processes as they are experienced in everyday circumstances. We used a semistructured interview guide, with specific close-ended prompts rather than an unstructured interview guide. This allowed us to use the same key questions for all interviews, but also to explore more deeply into this area, especially the barriers the current audit and feedback process presented to bedside clinicians in the ICU.

    Strengths of this study include involvement of all members of the ICU multidisciplinary team. This allowed us to approach the complexity and variation of audit and feedback implementation in the ICU to achieve the fullest understanding of this strategy. We also included both academic and community hospitals. In addition, we used a multicenter design and inclusion of 72 participants, which increases the credibility of our results as we have provided a large, inclusive and fuller perspective on a common and important quality improvement process in the ICU,6 which has not been described to date. Our study offers new insight into common dynamics that are encountered by many clinicians across several centres. From the ICU clinician perspective, this focus on their experience and understanding of audit and feedback captures a broad spectrum of the meanings and consequences attached to the use of this quality improvement strategy.

    Conclusions

    Greater attention to improving communication, integration of the process into daily clinical activities and making feedback timely, specific and actionable may influence the effectiveness of audit and feedback at changing desired behaviours. Future research could more fully explore the utility of customising clinical audit data so that it is meaningful to ICU clinician groups. To address the existing gaps in clinical practice, those in leadership positions at the unit level need to take steps to integrate feedback into the daily routine of the ICU and improve the timeliness and specificity of feedback.

    References

    1. ↵
      1. Adhikari NK,
      2. Rubenfeld GD
      . Worldwide demand for critical care. Curr Opin Crit Care 2011;17:620–5. doi:10.1097/MCC.0b013e32834cd39c
      OpenUrlCrossRefPubMed
    2. ↵
      1. Coopersmith CM,
      2. Wunsch H,
      3. Fink MP, et al
      . A comparison of critical care research funding and the financial burden of critical illness in the United States. Crit Care Med 2012;40:1072–9. doi:10.1097/CCM.0b013e31823c8d03
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. McMillan TR,
      2. Hyzy RC
      . Bringing quality improvement into the intensive care unit. Crit Care Med 2007;35(2 Suppl):S59–65. doi:10.1097/01.CCM.0000252914.22497.44
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Garland A
      . Improving the ICU: part 1. Chest 2005;127:2151–64. doi:10.1378/chest.127.6.2151
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Sinuff T,
      2. Cook DJ,
      3. Giacomini M, et al
      . Facilitating guideline adherence in the intensive care unit: a multicentre study. Crit Care Med 2007;35:2085–9.
      OpenUrl
    6. ↵
      1. Bosk CL,
      2. Dixon-Woods M,
      3. Goeschel CA, et al
      . The art of medicine: reality check for checklists. Lancet 2009;374:444–5. doi:10.1016/S0140-6736(09)61440-9
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Curtis JR,
      2. Cook DJ,
      3. Wall R, et al
      . Intensive care quality improvement: a “how-to” guide for the interdisciplinary team. Crit Care Med 2006;34:211–18. doi:10.1097/01.CCM.0000190617.76104.AC
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Ivers N,
      2. Jamtvedt G,
      3. Flottorp S, et al
      . Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012;6:CD000259. doi:10.1002/14651858.CD000259.pub3
      OpenUrlPubMed
    9. ↵
      1. Grimshaw JM,
      2. Thomas RE,
      3. MacLennan G, et al
      . Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004;8:iii–iv, 1–72. doi:10.3310/hta8060
      OpenUrlPubMed
    10. ↵
      1. Foy R,
      2. Eccles M,
      3. Jamtvedt G, et al
      . What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review. BMC Health Serv Res 2005;5:50. doi:10.1186/1472-6963-5-50
      OpenUrlCrossRefPubMed
    11. ↵
      1. Sinuff T,
      2. Muscedere J,
      3. Adhikari N, et al
      . Knowledge translation interventions for critically ill patients: a systematic review. Crit Care Med 2013;14:2627–40. doi:10.1097/CCM.0b013e3182982b03
      OpenUrl
    12. ↵
      1. Sinuff T,
      2. Eva K,
      3. Meade M, et al
      . Clinical practice guidelines in the ICU: a canadian survey. Can J Anaesthesia 2007;54:728–36. doi:10.1007/BF03026869
      OpenUrlCrossRefPubMed
    13. ↵
      1. Scales DC,
      2. Dainty K,
      3. Hales B, et al
      . A multifaceted intervention for quality improvement in a network of intensive care units: a cluster randomized trial. JAMA 2011;305:363–72. doi:10.1001/jama.2010.2000
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Abbott CA,
      2. Demsa T,
      3. Stewart DW, et al
      . Adoption of a ventilator-associated pneumonia clinical practice guideline. Worldviews Evid Based Nurs 2006;3:139–52. doi:10.1111/j.1741-6787.2006.00066.x
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Cocanour CS,
      2. Peninger M,
      3. Domonoske BD, et al
      . Decreasing ventilator-associated pneumonia in a trauma ICU. J Trauma 2006;61:122–9. doi:10.1097/01.ta.0000223971.25845.b3
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Jain MK,
      2. Heyland D,
      3. Dhaliwal R, et al
      . Dissemination of the Canadian clinical practice guidelines for nutrition support: results of a cluster randomized controlled trial. Crit Care Med 2006;34:2362–9. doi:10.1097/01.CCM.0000234044.91893.9C
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Elliott R,
      2. McKinley S,
      3. Aitken L
      . Adoption of a sedation scoring system and sedation guideline in an intensive care unit. J Adv Nurs 2006;54:208–16. doi:10.1111/j.1365-2648.2006.03798.x
      OpenUrlCrossRefPubMed
    18. ↵
      1. McMullin JP,
      2. Cook DJ,
      3. Griffith LE, et al
      . Minimizing errors of omission: behavioural rEenforcement of Heparin to Avert Venous Emboli (BEHAVE). Crit Care Med 2006;34:694–9. doi:10.1097/01.CCM.0000201886.84135.CB
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Creedon SA
      . Healthcare workers’ hand decontamination practices: compliance with recommended guidelines. J Adv Nurs 2005;51:208–16. doi:10.1111/j.1365-2648.2005.03490.x
      OpenUrlCrossRefPubMed
    20. ↵
      1. Rosenthal VD,
      2. Guzman S,
      3. Safdar N
      . Reduction in nosocomial infection with improved hand hygiene in intensive care units of a tertiary care hospital in Argentina. Am J Infect Control 2005;33:392–7. doi:10.1016/j.ajic.2004.08.009
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Brattebo G,
      2. Hofoss D,
      3. Flaatten H, et al
      . Education and debate. Quality improvement report: effect of a scoring system and protocol for sedation on duration of patients’ need for ventilator support in a surgical intensive care unit. BMJ 2002;324:1386–9. doi:10.1136/bmj.324.7350.1386
      OpenUrlAbstract/FREE Full Text
    22. ↵
      1. Merlani P,
      2. Garnerin P,
      3. Diby M, et al
      . Quality improvement report: linking guideline to regular feedback to increase appropriate requests for clinical tests: blood gas analysis in intensive care. BMJ 2001;323:620–4. doi:10.1136/bmj.323.7313.620
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Price J,
      2. Ekleberry A,
      3. Grover A, et al
      . Evaluation of clinical practice guidelines on outcome of infection in patients in the surgical intensive care unit. Crit Care Med 1999;27:2118–24. doi:10.1097/00003246-199910000-00007
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Sinuff T,
      2. Cook DJ,
      3. Peterson JC, et al
      . Development, implementation, and evaluation of a ketoconazole practice guideline for ARDS prophylaxis. J Crit Care 1999;14:1–6. doi:10.1016/S0883-9441(99)90001-1
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Donabadian A
      . Explorations in quality assessment and monitoring. Ann Arbour, MI: Health Administration Press, 1980.
    26. ↵
      1. Jamtvedt G,
      2. Young JM,
      3. Kristoffersen DT, et al
      . Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2006;(2):CD000259.doi:10.1002/14651858
    27. ↵
      1. Hysong SJ
      . Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care 2009;47:356–63. doi:10.1097/MLR.0b013e3181893f6b
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Gardner B,
      2. Whittington C,
      3. McAteer J, et al
      . Using theory to synthesize evidence from behavior change interventions: the example of audit and feedback. Soc Sci Med 2010;70:1618–25. doi:10.1016/j.socscimed.2010.01.039
      OpenUrlCrossRefPubMed
    29. ↵
      1. Hysong S,
      2. Teal CR,
      3. Khan MJ, et al
      . Improving quality of care through improved audit and feedback. Implement Sci 2012;7:45. doi:10.1186/1748-5908-7-45
      OpenUrlCrossRefPubMed
    30. ↵
      1. Denzin NK,
      2. Lincoln YS
      . Handbook of qualitative research. 2nd edn. Sage Publications, 2000.
    31. ↵
      1. Strauss A,
      2. Corbin J
      . Basics of qualitative research. Techniques and procedures for developing grounded theory. 2nd edn. London: Sage Publications, 1998.
    32. ↵
      1. Michie S,
      2. Johnston M,
      3. Abraham C, et al
      . “Psychological Theory” Group. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care 2005;14:26–33. doi:10.1136/qshc.2004.011155
      OpenUrlAbstract/FREE Full Text
    33. ↵
      1. Cane J,
      2. O'Connor D,
      3. Michie S
      . Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci 2012;7:37. doi:10.1186/1748-5908-7-37
      OpenUrlCrossRefPubMed
    34. ↵
      1. Damschroder LJ,
      2. Aron DC,
      3. Keith RE, et al
      . Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4:50. doi:10.1186/1748-5908-4-50
      OpenUrlCrossRefPubMed
    35. ↵
      1. Sanndelowski M
      . Whatever happened to qualitative description? Res Nurs Health 2000;23:334–40. doi:10.1002/1098-240X(200008)23:4<334::AID-NUR9>3.0.CO;2-G
      OpenUrlCrossRefPubMedWeb of Science
    36. ↵
      1. Sandelowski M
      . What's in a name? Qualitative description revisited. Res Nurs Health 2010;33:77–84.
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      1. Pronovost PJ,
      2. Angus DC,
      3. Dorman T, et al
      . Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. JAMA 2002;288:2151–62. doi:10.1001/jama.288.17.2151
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Sandelowski M
      . The problem of rigor in qualitative research. ANS Adv Nurs Sci 1986;8:27–37. doi:10.1097/00012272-198604000-00005
      OpenUrlCrossRefPubMed
    39. ↵
      1. Kluger AN,
      2. DeNisi A
      . The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull 1996;119:254–84. doi:10.1037/0033-2909.119.2.254
      OpenUrlCrossRefWeb of Science
    40. ↵
      1. Bradley EH,
      2. Holmboe ES,
      3. Mattera JA, et al
      . Data feedback efforts in quality improvement: lessons learned from US hospitals. Qual Saf Health Care 2004;13:26–31. doi:10.1136/qhc.13.1.26
      OpenUrlAbstract/FREE Full Text
    41. ↵
      1. Deshon RP,
      2. Kozlowski SW,
      3. Schmidt AM, et al
      . A multiple-goal, multilevel model of feedback effects on the regulation of individual and team performance. J Appl Psychol 2004;89:1035–56. doi:10.1037/0021-9010.89.6.1035
      OpenUrlCrossRefPubMedWeb of Science
    42. ↵
      1. Archer-Kath J,
      2. Johnson DW,
      3. Johnson RJ
      . Individual versus group feedback in cooperative groups. J Soc Psychol 1994;134:681–94. doi:10.1080/00224545.1994.9922999
      OpenUrlCrossRefWeb of Science
    43. ↵
      1. Hysong SJ,
      2. Best RG,
      3. Pugh JA
      . Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci 2006;1:9. doi:10.1186/1748-5908-1-9
      OpenUrlCrossRefPubMed
    44. ↵
      1. Leung K,
      2. Su S,
      3. Morris MW
      . When is criticism not constructive? The roles of fairness perceptions and dispositional attributions in employee acceptance of critical supervisory feedback. Hum Relat 2001;54:1155–87. doi:10.1177/0018726701549002
      OpenUrlCrossRef
    45. ↵
      1. Kinicki AJ,
      2. Prussia GE,
      3. Wu B, et al
      . A covariance structure analysis of employees’ response to performance feedback. J Appl Psychol 2004;89:1057–69. doi:10.1037/0021-9010.89.6.1057
      OpenUrlCrossRefPubMedWeb of Science
    46. ↵
      1. Locke EA,
      2. Latham GP
      . Building a practically useful theory of goal setting and task motivation: a 35-year odyssey. Am Psychol 2002;57:705–17. doi:10.1037/0003-066X.57.9.705
      OpenUrlCrossRefPubMedWeb of Science
    47. ↵
      1. Locke EA
      . Goal theory vs. control theory: contrasting approaches to understanding work motivation. Motiv Emotion 1991;15:9–28. doi:10.1007/BF00991473
      OpenUrlCrossRef
    48. ↵
      1. Carver CS,
      2. Scheier MF
      . Control theory: a useful conceptual framework for personality-social, clinical and health psychology. Psychol Bull 1998;92:111–35. doi:10.1037/0033-2909.92.1.111
      OpenUrl
    49. ↵
      1. Dale C,
      2. Fowler RA,
      3. Adhikari NK, et al
      . Implementation of a research awareness program in the critical care unit: effects on families and clinicians. Intensive Crit Care Nurs 2010;26:69–74. doi:10.1016/j.iccn.2009.09.003
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Correction notice This article has been corrected since it was published online first. The third author's surname has been corrected.

    • Acknowledgements We would like to thank all of the participants for their invaluable time and contribution to the interviews comprising this study.

    • Contributors TS and LR had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: TS, JM and DCS. Acquisition of data: LR. Analysis and interpretation of data: LR, TS and CMD. Drafting of the manuscript: TS, JM, LR, DCS and CMD. Critical revision of the manuscript for important intellectual content: TS, JM, LR, DCS and CMD. Obtained funding: TS, JM and DCS.

    • Funding This study was funded by the Physicians' Services Incorporated Foundation (PSI), University of Toronto Dean's Fund, and the University of Toronto Connaught Fund. DCS is supported by a Graham Farquharson Knowledge Translation Fellowship from PSI.

    • Competing interests None declared.

    • Ethics approval Research Ethics Boards of each of the participating institutions.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement All data from the study are included in the submitted manuscript.