• Healthcare quality improvement
• Checklists
• Failure Modes and Effects Analysis (FMEA)
• PDSA

The practice and reporting of healthcare improvement has matured significantly since its origin early this century. Most initial improvement reports were akin to management reports or case studies. They often lacked the specificity and details needed to communicate the design, performance and findings with enough precision, accuracy and thoroughness to allow readers to assess validity of the design and execution of the work.

Prevention of central line-associated bloodstream infection (CLABSI) has yielded some of the field's most informative ‘foundation stories,’ illustrating the successes and challenges of identifying interventions that work, why they work and how to make them work in different settings.1 ,2 An important lesson learned from attempts to replicate potentially generalisable successes, such as checklists for CLABSI,3 was that the behaviours, attitudes, alliances and cultural shifts that occasion, and are occasioned by, the act of improvement are more complex4 and more powerful than what might be termed the ‘hardware’ of the improvement—the checklist, template or scripted procedure itself.

The article by Dandoy et al5 adds an additional chapter to the developing story of healthcare improvement practice and reporting by relating one group's response to an observed failure of a previously successfully CLABSI-prevention bundle. The description of this highly developed group's investigation and rapid-cycle improvement response to changes in previously stable CLABSI rates suggests potentially translatable improvement ‘hardware,’ such as requiring two people for dressing changes or requiring baths, oral care and out-of-bed activities. However, the mechanisms by which these components may have led to the measured outcomes remain unclear: did the components lead to improvement, or were they just part of the complex solution generated by a culture of improvement? By occasioning this question, the article demonstrates some of the major challenges inherent in reporting and disseminating improvement work.

The initial Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines in 2008 reflected the need to formalise writing about Quality Improvement (QI) in order to support knowledge transfer.6 As our collective experience has grown in the field, so has our understanding of three main areas of development in healthcare improvement. The first area is the use of a theoretical framework or rationale explaining why an intervention was expected to be effective in a particular context.7 The second, which overlaps with the first, concerns how the local context impacts the success of an intervention and the intervention itself. The third concerns how we study the intervention and the performance of the intervention. The updated SQUIRE 2.0 guidelines8 reflect this evolution. Close reading—indeed, interpolative reading—of the improvement report by Dandoy et al sheds some light on how to navigate these areas as we endeavour to develop generalisable and actionable knowledge in healthcare improvement.

In response to the rising CLABSI rates, the team engaged in a failure mode and effects analysis (FMEA), to identify patient, provider and environmental factors associated with increased infections. Although the authors do not explicitly state the rationale informing their interventions, they narrate how the FMEA investigation into causes informed the intervention design, specifically to alleviate system stressors, to change provider behaviours in maintaining central lines and to modify some patient factors. The rationale for causation is implied in the assessment–intervention format they employ in reporting their intervention.

The methods section provides details on the setting in which the intervention occurred and can be read as an examination, through the FMEA approach, of forces in the environment which were believed to contribute to increasing infections. While it has become fairly common in quality improvement reports to include descriptions of the setting, and to conflate this description with context, context goes conceptually far beyond the intervention's setting.9 It includes a diversity of determinants of how healthcare service is performed, including factors such as organisational culture, extent of interprofessional collaboration, institutional track record with improvement projects and a variety of other factors laid out in various taxonomies.10–12 Reading between the lines, we assume the many contextual elements that must have been present: robust informatics, strong leadership support, availability of necessary personnel, a culture of multidisciplinary care, financial resources to adjust staffing and so on. For a reader interested in applying the intervention in her own setting, these contextual elements are important.

The intervention has a remarkable number of components, making it difficult to know what aspects were most important, the degree of implementation of each, or the extent to which they were sustained. Studying an intervention is challenging when the focus is on ‘solving a problem’, which in this example (and often in improvement work) seems to encompass a ‘throw everything at it and see if it sticks’ approach. This approach differs markedly from the evaluation of a single intervention implemented in a stepwise approach. One example where this can be seen from within this complex intervention is compliance with the 1–2–3 initiative. This thoughtful component of the overall intervention required significant effort and could, in and of itself, have been the subject of an improvement report. The multidisciplinary 1–2–3 initiative required cooperation from patients and families to increase hygiene and physical activity. The authors present a figure that illustrates a fairly rapid increase in adherence rates per opportunity from 25% to 70%, but it is not clear how these were observed and assessed, how the authors were able to conclude that the ‘greatest impact on compliance’ was related to text message reminders, and how family members were engaged. As a result, readers might have more difficulty replicating and evaluating this specific intervention in their own settings.

This perhaps overly detailed critique highlights an ongoing challenge in improvement reporting—where does comprehensive reporting end and exhaustive cataloguing of minutiae begin? As we now better appreciate context, the goal of ‘replication’ of the improvement processes and the contextual conditions in an effort to achieve identical results is proving to be an unintelligible goal. A more robust goal is to narrate the general process of improvement and report representative outcomes. Reproducibility in healthcare improvement might be best thought of as emulation: seeking to match or surpass an achievement through imitation. Emulating the work described in an improvement report is a decidedly different task from replicating or reproducing it. The intervention ‘hardware’ might be duplicated; the context cannot.

A laudable component of the report by Dandoy et al5 is that the authors resist the temptation (often an irresistible temptation in the early ‘checklist school’ of QI) to condense their success into a simple formula. The narrative of their work is most informative from a ‘meta’ perspective, reading the authors as characters within their own context. What we learn is that an experienced, high functioning team, with an established, advanced CLABSI bundle in place, can experience low-quality outcomes when faced with external forces that stress the environment. This same high functioning team is subsequently able to respond rapidly to this stress through deductive and inductive investigation. Then, using resources that are indicative of institutional support and the existing culture of improvement, they develop new and appropriate responses that reduce infections. This provides an example to understand deeply how entwined rationale, context and the study of the intervention are in improvement cycles. This also demonstrates the necessary role of each in considering how an intervention might be emulated in another context, or to address a similar challenge.

The important story that Dandoy et al5 contribute is ultimately the story that gets told implicitly: that of the power of context in determining response to changes in performance. For those seeking empiricism, the task of improvement storytelling can seem daunting, but doing it well acknowledges that the crucial problems of improvement work are often complex, and may be better approached through narrative as opposed to paradigmatic thinking.1 The challenge of translating this narrative thinking into writing, and distinguishing and clarifying rationale, context and the study of the intervention, may bring to mind John Muir's assertion that ‘when we try to pick out anything by itself, we find it hitched to everything else in the Universe’.13 These new elements, as outlined in SQUIRE 2.0, provide direction in that challenge. Understanding and influencing the interrelationships that determine how healthcare is produced remains one of the great frontiers.