Since the seminal studies by Gawande and colleagues1 and Pronovost et al,2 checklists have become the go-to solution for a vast range of patient safety and quality issues in healthcare. Some see them as a quick and obvious solution to a relatively straightforward problem. For others, they illustrate a failure to understand and address the complex challenges in patient safety and quality improvement. Indeed, successes3 and failures4–6 illustrate an underlying difficulty with understanding precisely why checklists work in some cases but not in others. A recent viewpoint summarises the varying applications of checklists in aviation and healthcare, reflecting upon the dangers of making assumptions about their ‘ubiquitous utility’.7 This provided a timely “The Problem with…”8 opportunity, in which we consider the narratives that often surround the challenges faced in designing and implementing a successful checklist, and the science used to explore it.

‘A simple idea from other industries…’

The apparent simplicity of a checklist is understandingly tempting, with some narratives suggesting that their adoption can be used to effectively address what would appear to be intractable, complex and potentially painful systems issues. However, this simple narrative does not always reflect an understanding of the problems needing to be solved, how best to solve them or indeed the intricacies surrounding the implementation, use and impact of such a simple looking tool. More likely, what we face in introducing a checklist is a rather more complex story of gains and losses, procedural interactions and sociocultural balances (see table 1). This ‘simple’ versus ‘complex’ narrative can also be seen in the frequent aviation analogies, which imply that checklists prevented accidents (‘simple’), while omitting to mention the critical design changes that were also necessary (‘complex’). For example, on the B-17 aircraft, flap and gear levers required redesign as they were easily confused, critically positioned and thus predisposing to accidents.10

We have often compared healthcare checklists and their evidence base with checklists used in other industries, but there are some important qualitative differences. For example, as the companion viewpoint also observes, healthcare checklists do not always share design features with their aviation counterparts. For an Airbus A319 (figure 1), a single laminated gatefold (four sides of normal A4 paper) contains the 13 checklists for normal and emergency operations. Tasks range from 2 (for cabin fire checklist) to 17 (for before take-off checklist), with an average of seven per checklist. Each task is described in no more than three words and can be checked immediately, with usually a single word of confirmation. It has no check boxes, does not require signature and is designed to be used by one person, with specific checklists performed aloud. In contrast, the Centers for Disease Control and Prevention central line-associated blood stream infections checklist11 has 18 tasks, with no less than 4 word descriptors (and up to 22 words), and describes non-procedural tasks that need to be completed over several minutes (and hours), which cannot be ‘checked’ (eg, ‘empower staff’). The WHO safer surgery checklist (first edition)12 has 21 tasks (7+7+7), with wording ranging from 2 to 16 per task, and involves several people simultaneously. Some tasks are easily checked and completed, while some require discussion and some cannot be ‘checked’.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Detail of ‘Normal’ Checklists for Airbus A319 (United Airlines, 19 December 2003).

One feature of checklists in healthcare, in comparison to most other industrial uses, is that they increasingly feature items intended to promote communication and teamwork (eg, introductions, discussion of patient risk factors, concerns and so on), in addition to straightforward categorical checks (eg, have hands been washed, has consent been obtained and others). If used in the right way, they can indeed assist in the change of communication patterns and specific coordination tasks (such as ‘call outs’, task allocation and task visibility).10 However, creating an opportunity for a more general team talk is not a traditional feature or necessarily a particular strength of checklists. In fact, authentic checklist completion will rely on good communication and teamwork in the first place, which is not always the case. This use as a ‘teamwork and communication tool’ therefore goes beyond the classic uses of an ‘aviation style’ checklist.

Team-related checklists have further difficulties because there is often no single calm moment where everyone can be involved, and not every user will experience a direct benefit despite the effort and initiative required for their use. Furthermore, the more complex sociological and cultural challenges, such as power distance, hierarchy and perceptions of professionalism, will often continue to dominate interactions regardless of the design and implementation of the checks. Introducing a checklist reliant on communication without consideration of these processes undermines the sociocultural underpinnings of the intervention and the complexity of coordinating a complex team around a single task.

Thus, prominent uses of checklists in healthcare, such as the surgical safety checklist and the central line bundle, are far from ‘simple’ and are not that similar to applications of checklists in the industries that inspired their use.

‘…That leads to better outcomes’

Successful replication of interventions requires us to know why and how they work, that is, to understand the mechanism of effect.13 A checklist will support a well-defined procedural task that may be vulnerable to problems with memory, task omissions or order. Success may be due to the specific items on the checklist, the non-specific team talking that is promoted, or some other artefacts of the intervention. It is not always clear which is the most important, or how this varies according to context. If it is the ‘team talk’ that is important, then the items on the checklist and, indeed, the checklist itself, may be inconsequential.14 Alternatively, if the successful use of a checklist requires the systematic completion of every item, then efforts to ensure compliance should be directed towards the appropriate checking of each item. However, a checklist reliant on teamwork for success may fail despite all the items being followed, because those team skills were insufficient. This can lead to compliance and audit measures that may unfairly penalise (not ‘checking all the boxes’) while discouraging appropriate use (having a team discussion). Thus the ‘evidence’ for ‘checklists’ can quickly become a semantic or mechanistic misattribution and a correlation/causation fallacy.

This confusion with mechanism and assurance is superimposed on the existing dichotomy of the checklist as a ‘rule’—which requires everyone to use the checklist, methodically, in the same way, or as an ‘aid’—where it may be used differently by experts who are already reliable and proficient. Without direct observation of checklist use, therefore, it is at least open to significant challenge that observed outcomes are related to checklist use or, alternatively, that full compliance may not be necessary to achieve the desired effect. Future studies must seek to address the level and type of compliance (which is unlikely to reside at 100%) required to achieve the observed outcome effects, and understand and define the mechanisms of effect.

The failure to replicate results in healthcare4 ,5 demonstrates that at the very least, where success has been observed, it was not only a checklist that created the effect.15 Much of the variation in reported impact has been attributed to the style of implementation employed. While we are coming to acknowledge that simply presenting a checklist to users and enforcing strict progression through the items upon it may not necessarily yield performance or outcome benefits, optimal checklist implementation approaches are becoming notoriously difficult to define in healthcare. Favourable implementation conditions are well published (eg, education, local modification, feedback, senior support),16 however, still do not reliably lead to positive checklist effects.5 This makes it difficult to draw lessons from the international evidence base that will work at a local level.

Just as checklists were necessary, but not sufficient, in aviation, it seems probable that viewing checklists as an adequate intervention to address the complexities of modern-day medicine in isolation is a somewhat naive viewpoint. More likely, their integration into broader multifaceted team-training ‘packages’ that incorporate additional team-based interventions (eg, briefings and de-briefings), include a focus on culture and support the skills required to complete the process will likely create a milieu in which a well-designed checklist can work.14

Lost in translation?

There is no question that the right checklist, in the right place, with the right design and implementation, can be used enthusiastically by the right people with the right skills and can be highly effective. Yet, in translating checklists from aviation and other industries to healthcare, we may have misunderstood their strengths, failed to design them based on well-established principles and failed to engineer them as a component of a wider socio-technical system. We have made assumptions about their use, effectiveness and ‘evidence base’ that are readily and easily challenged, and have defined compliance criteria and penalties based on assumptions that may not reflect how they contribute to better outcomes. The superimposition of teamwork and communication—without specifically providing training for those skills, or indeed the sociocultural support for them—further contributes to the difficulties in successful implementation.

A checklist is a complex socio-technical intervention17 ,18 that requires careful attention to design, implementation and basic skills required for the task.19 Understanding and specifying these mechanisms of effect with greater precision would enable us to move beyond the moot ‘checklists do/don’t work’ commentaries. Much as we would like it to be true, the story that checklists are a ‘simple, evidence-based solution successfully used in other industries’ has many of the characteristics of ‘Cargo Cult’ or pseudoscience.20 There is indeed a science to checklists. But, unless we pay attention to the more complex narrative for how they emerged in other industries, including the other changes (to culture, teamwork and design) that accompanied them, we stand little chance of appreciating that science or realising similar benefits in healthcare.