Central venous catheter-related bloodstream infection (CVC-BSI) is common, costly and potentially fatal.1 For decades, conventional wisdom regarded these infections as inevitable complications of care. This view changed following landmark studies2 ,3 that demonstrated substantial reductions associated with the CVC-BSI ‘bundle.’4 ,5

Investigators at Johns Hopkins University designed an improvement model that featured (1) a checklist, or bundle, of evidence-based practices (proper hand hygiene, chlorhexidine for skin antisepsis, use of maximal sterile barriers, avoidance of the femoral site); (2) education regarding these infection-control practices; (3) a catheter-insertion cart; (4) daily review and prompt removal of unwarranted CVCs and (5) empowerment of nurses to enforce adherence to these practices.3

An initial evaluation of this bundled intervention revealed an impressive decrease in CVC-BSIs from 11.3 infections/1000 catheter days to 0/1000 catheter days at Johns Hopkins. Seeking external validation, investigators partnered with the Michigan Keystone Health and Hospital Association to evaluate the CVC-BSI bundle in 103 intensive care units (ICUs) across 77 hospitals.6 This study again showed a large and statistically significant reduction in CVC-BSIs, from a baseline mean of 7.7 infections/1000 catheter days to 1.4/1000 catheter days. The CVC-BSI bundle had arrived.

The success of the CVC-BSI bundle stimulated interest in checklists for surgical safety.7 WHO's Surgical Safety Checklist led to substantial improvements in operative outcomes in diverse clinical settings.8 A study of multiple checklists at different stages in the perioperative period showed impressive improvements in surgical complications and mortality at six hospitals in The Netherlands.9 These dramatic results—in ICUs and operating rooms—made checklists virtually synonymous with safer innovative care.

How may checklists work?

Early in the checklist movement, some investigators suggested that treating checklists as ‘tick-box’ exercises may lead the field astray. They argued that checklists contain not just technical elements, but also ‘socioadaptive’ ones.10 ,11 Technical elements, such as the use of chlorhexidine for skin antisepsis or administration of antibiotics 30–60 min prior to skin incision, comprise discrete, easily implemented actions. Socioadaptive elements (eg, removal of unnecessary CVCs, or the preoperative time-out, to discuss critical steps in a surgical plan) involve more than simple actions: they require true engagement in the tasks. Implementation efforts must, therefore, address teamwork, communication and culture.

Recognising the importance of socioadaptive changes in unlocking the benefits of checklists served a vital purpose for the field. Yet, not all checklists require the same attention to socioadaptive elements. At one end of the spectrum, most items on the surgical checklist require improved teamwork and communication. The preoperative time-out to facilitate introductions between members of an operative team, verify the surgical site, and review critical steps, will achieve no benefit if performed merely as tick-box exercises.12 Administration of perioperative antibiotics, however, constitutes a notable exception. When administered as recommended, perioperative antibiotics reduce surgical site infections irrespective of any special communication between anaesthetists, surgeons and nurses.

By contrast, hospital admission order sets consist almost entirely of tick-box-style technical elements. A typical order set offers standard orders for monitoring vital signs, diet, laboratory investigations and perhaps thromboembolism prophylaxis. As long as clinicians tick the boxes, each order will achieve its intended goal.

The CVC-BSI bundle lies somewhere between these two extremes, blending purely technical items with ones that require socioadaptive changes. Consistent use of chlorhexidine gluconate requires a commitment from hospital leadership to purchase this agent instead of povidone-iodine. Achieving this commitment may initially call for socioadaptive measures. Once accomplished, however, physicians will necessarily use chlorhexidine for skin disinfection as it will be the only antiseptic available. The socioadaptive behaviours required to successfully implement chlorhexidine, thus occur upstream to checklist deployment. This phenomenon occurs with order sets as well. Convincing clinicians to use an order set, obtaining agreement on the included elements, and selecting a single anticoagulant for thromboembolism prophylaxis all also require socioadaptive behaviours. However, once enacted, these technical elements exert their effects without further attention to communication, teamwork or culture.

Some aspects of the CVC-BSI bundle do require ongoing socioadaptive behaviours. Nurses must feel comfortable pointing out non-compliance with full barrier precautions, and physicians must heed nurses’ reminders to do so. Similarly, prompt removal of unnecessary CVCs requires physicians to evaluate the necessity of central venous access and solicit inputs from nurses on this point. Recognising the degree to which checklists differ in their dependence on socioadaptive elements is important for two reasons. First, the variation in the ‘active ingredients’ of checklists (technical elements alone vs technical plus socioadaptive ones) underscores the importance of theory in developing and evaluating patient safety interventions.13 ,14 Second, while improving teamwork and culture holds clear appeal, the active ingredients of some checklists may consist entirely of specific technical elements. The use of chlorhexidine and a full sterile drape, by themselves, produce reductions in CVC-BSIs comparable with those reported for the CVC-BSI bundle.15–17 Any accompanying changes in teamwork and communication that occur during implementation may simply represent epiphenomena.

The active ingredients of the CVC-BSI checklist

This issue of the journal reports the findings of ‘Matching Michigan,’ a national UK initiative to reduce CVC-BSIs.18 Matching Michigan aimed to reduce CVC-BSI to at least the mean rate achieved in Michigan (1.4 infections/1000 CVC patient days).6 This two-year, prospective, controlled trial allocated 223 adult and paediatric ICUs in the UK to four clusters that joined the programme at different phases (ie, a ‘stepped design’). The first three clusters were defined by geographic region, while the fourth consisted of ICUs unable to join the project in an earlier phase.

The desired decline in CVC-BSIs occurred, with a decrease from 4.4 to 1.7 CVC-BSIs/1000 patient days. However, sequentially enrolled ICU clusters exhibited CVC-BSI rates that declined on par with the rates achieved in ICUs already participating in the intervention. Furthermore, CVC-BSIs occurring within 48 h of admission showed comparable declines. These two findings—reduced CVC-BSI rates at the time of joining the intervention and reductions in early CVC-BSIs not related to ICU care—led the authors to infer secular trends as responsible for reduced CVC-BSI, rather than intervention effects.

The authors offer several additional explanations for their findings, including heightened awareness of CVC-BSI and multiple national reforms aimed at curbing this problem. Certainly, the lower baseline rate of CVC-BSI in Matching Michigan, compared with that of Michigan-Keystone, bears out the effects of existing initiatives. Nonetheless, a further reduction in CVC-BSI of approximately 60% still occurred during the study, an effect size comparable with that observed in Michigan.6 Thus, it was not the case that performance was already too high to permit further improvement; rather, improvement occurred in most sites irrespective of their status in the study.

Two hypotheses may explain these findings. First, technical elements, such as chlorhexidine for skin disinfection and hand hygiene, may deliver much of the benefit of the CVC-BSI checklist. US data suggest a close association between the use of technical interventions and CVC-BSI control.19 Data from England are sparse, but support this association.20 ,21 Technical elements have robust effects,16 are easily implemented and are cost-effective.17 Adoption of these practices outside of the study may completely account for the observed national decline in CVC-BSIs.

Successful implementation of most improvement interventions requires attention to teamwork, culture and other socioadaptive behaviours. So, the rare cases in which improved outcomes occur on the basis of simple technical elements should count as good news. Unfortunately, variation in approaches for detecting CVC-BSI at participating sites may also explain the results of Matching Michigan.

CVC-BSI measurement: an illusion of objectivity

Measurement presents a major problem when it comes to CVC-BSI.22 ,23 The clinical decision to obtain blood cultures directly impacts BSI rates. ICUs that obtain more blood cultures will inevitably document more CVC-BSIs.24 In addition, surveillance CVC-BSI definitions stipulate absence of other sources of infection to explain positive blood cultures. The degree to which an alternate source of infection could explain a positive blood culture, however, also involves subjective judgment.25 These criterion uncertainties explain discrepancies between reported and audited rates of CVC-BSI.24 ,26

The authors of the parallel ethnographic study27 of Matching Michigan emphasise the absence of data manipulation or gaming behaviours on the part of staff in participating ICUs. We do not doubt this impression. The picture of CVC-BSI data that emerges is not one of deliberate deception, but of an outcome that says more about attitudes and data-collection practices than about actual event rates. As the authors concluded, ‘Rather than objective measures of incidence, differences in reported infection rates may reflect, at least to some extent, underlying social practices in data collection and reporting and variations in clinical practice.’27

This situation resembles that of incident reporting: the frequency of critical incidents says more about willingness to report than about underlying harm.28 For this reason, increases in critical incidents are widely acknowledged to indicate improvements in institutional culture rather than increased harm to patients. Similarly, CVC-BSI rates may say more about the assiduousness with which clinicians look for this outcome (do they obtain blood cultures only in the face of obvious signs of infection or even for less obvious cases?). Thus, declines in CVC-BSI rates may reflect decreased likelihood of obtaining blood cultures rather than true declines in infections.

A diminished propensity to obtain blood cultures in the setting of an intervention to reduce CVC-BSIs makes psychological sense even in the absence of gaming. If clinicians have taken steps to reduce a complication, they will less often suspect the occurrence of this complication. A similar phenomenon occurs with venous thromboembolism prophylaxis. When hospitalised patients develop unexplained shortness of breath, clinicians routinely consider the possibility of pulmonary embolism; but, they will only test for this diagnosis in the most obvious cases if patients are consistently receiving pharmacologic thromboembolism prophylaxis. Thus, evaluation of an intervention to improve venous thromboembolism prophylaxis that relied on the decisions of clinicians to pursue the diagnosis would show improvements in the outcome simply on the basis of decreased testing.

For venous thromboembolism prophylaxis, fairly robust evidence supports true improvements, rather than just measurement effects.29 For the CVC-BSI bundle, by contrast, it seems plausible to infer altered measurement, rather than intervention effects or even marked secular trends. This explanation seems particularly likely in light of the fact that technical elements of the CVC-BSI bundle were already in place in most ICUs enrolled in Matching Michigan. Moreover, few ICUs adopted non-technical elements, making further declines throughout the 2-year study improbable. The fact that pre-ICU infections declined on par with ICU infections, and control ICUs exhibited the same CVC-BSI rate as intervention ICUs from the moment they joined the programme further underscores the possibility of a fundamental measurement problem.

The way forward for checklists

What, then, to take away from this landmark study? Matching Michigan highlights the need to determine how, not just if, checklists improve safety. While improving culture and teamwork holds understandable attraction, the effects of the CVC-BSI checklist may reflect something as simple—and as powerful—as using chlorhexidine and hand hygiene. We do not associate changes in teamwork and communication with the effectiveness of thromboembolism prophylaxis in an order set. Similarly, socioadaptive elements may not moderate the effects of the CVC-BSI bundle. Matching Michigan demonstrates how we need not only rigorous evaluations that document the effects of complex interventions, but also studies that identify the active ingredients of such interventions.

Second, we must improve outcome measurement in patient safety. Many outcomes in the field, not just CVC-BSI, involve fundamentally subjective elements—ventilator-associated pneumonia,30 preventable adverse events, ‘unexpected’ cardiac arrests in the setting of evaluations of rapid response teams, ‘unintended medication discrepancies’ and diagnostic errors, to name but a few. Social and psychological factors at the individual and institutional levels make accurate ascertainment of such outcomes, at the very least, a complex challenge, if not an unachievable goal. Technology-based innovations using objective data-derived elements to evaluate the true impact of safety interventions are urgently needed.

Finally, the sterility of a randomised controlled trial is a luxury most real-world improvement interventions cannot provide. That said, Matching Michigan demonstrates how rigorous, non-randomised designs supplemented by qualitative evaluation provide rich and robust data. These evaluative designs represent a major advance over the uncontrolled before/after study, still so woefully common in quality improvement reports. In this regard, Matching Michigan will remain a model in the years ahead of how large, well-designed studies advance the science of patient safety.