Statistics from Altmetric.com
Although surgery represents a mainstay of medical treatment, in industrialised countries, the rate of perioperative death directly due to inpatient surgery has been estimated at 0.4–0.8%, and the rate of major complications has been estimated at 3–17%.1 ,2 These complications include wrong patient/procedure/site surgery, anaesthesia equipment problems, lack of availability of necessary equipment, unanticipated blood loss, non-sterile equipment, and surgical items (eg, sponges) left inside patients. The complexity of most surgical procedures requires a well coordinated team to prevent these events.
Strategies for patient safety
Surgical checklists can potentially prevent errors and complications which may occur during surgery or perioperatively. A variety of interventions have shown promise for improving patient safety. For instance, Neily et al3 found that surgical team training which incorporated surgical checklists along with communication strategies was associated with a significant reduction in surgical mortality. Arriaga et al4 found that checklists dramatically improved adherence to critical processes of care in simulated scenarios of surgical crises. Studies have suggested that checklists may reduce errors for many reasons, including ensuring that all critical tasks are carried out, encouraging a non-hierarchical team-based approach, enhancing communication, catching near misses early, anticipating potential complications, and having technologies to manage anticipated and unanticipated complications. The WHO Surgical Safety Checklist is a prominent example of a surgical checklist intended to ensure safe surgery and minimise complications. Launched in June 2008, it has been translated into at least six languages.5 The 2009 WHO checklist (http://www.who.int/patientsafety/safesurgery/en/) contains 22 items in three phases:
Before induction of anaesthesia, covering areas such as patient identification, anaesthesia equipment check and a pulse oximetry check.
Before skin incision, covering areas such as team introductions, review of critical steps and antibiotic prophylaxis.
Before patient leaves operating room (OR), covering areas such as checking counts of instruments, specimen labelling and concerns for recovery.
In this paper we discuss the evidence for three patient safety efforts associated with surgical checklists. The WHO Surgical Safety Checklist and the Joint Commission Universal Protocol (UP) for Preventing Wrong Site, Wrong Procedure, Wrong Person Surgery6 have each been widely implemented to improve care when surgical procedures are performed. We also discuss the Surgical Patient Safety System (SURPASS) checklist,7–10 which represents a more comprehensive approach, capturing clinical care from admission to surgery to discharge.
We conducted a systematic literature search of MEDLINE, CINAHL, EMBASE and the Cochrane Database of Controlled Trials using a search strategy developed by a medical librarian. The search strategy (available upon request) included studies published from 1 January 2000 to 26 October 2012, and used a combination of medical subject headings and keywords related to checklists (‘anaesthesia checklist’, briefing, checklist, checkout, communication, documentation, instrument, ‘safety checklist’, tool, ‘surgical checklist’, protocol, ‘WHO checklist’).
Given the limited scope of this review, we focused on any articles describing actual use of the WHO checklist, the SURPASS checklist, a wrong-site surgery checklist or anaesthesia equipment checklists. We recognise other surgical checklists exist; however, many of these have only been implemented at a single institution. We also included articles describing use of anaesthesia checklists to detect equipment failure in simulated scenarios. This manuscript summarises all but the anaesthesia equipment checklists, which are described in an Agency for Healthcare Research and Quality (AHRQ) publication.11 An overview of the three types of checklists discussed in this paper is given in table 1. We included a total of 33 studies of these checklists and tabulated the reported outcomes, facilitators and barriers to checklist implementation.
Benefits and harms
The 2008 WHO Surgical Safety Checklist was tested at eight sites around the world.5 These settings varied greatly in the number of beds (range 371–1800), the number of ORs (range 3–39), and the income level of the country (four low, four high). Surgical safety policies prior to implementation of the WHO Checklist also differed regarding the use of routine intraoperative monitoring with pulse oximetry (six of eight sites), oral confirmation of patients’ identity and surgical site in the OR (only two of eight sites), and routine administration of prophylactic antibiotics in the OR (five of eight sites). None of the eight sites had a ‘standard plan for intravenous access for cases of high blood loss’, or formal team briefings preoperatively or postoperatively.
Baseline data were obtained at each site for 3 months prior to checklist introduction, involving a total of 3733 surgical procedures. In the subsequent 3–6-month period after checklist introduction, involving 3955 procedures, data showed decreases in patient mortality (from 1.5% to 0.8%) and inpatient complications (from 11% to 7%). No single site was driving the findings, as evidenced by the persistence of findings after the removal of any single site in a sensitivity analysis. The authors found that the performance rates for six specific safety indicators (eg, using a pulse oximeter) also increased after checklist introduction, suggesting that the safety indicators may have been responsible for the lower rates.
In discussing the results, the authors acknowledged that the underlying explanations were ‘most likely multifactorial’ and included the following:
The checklist itself.
A Hawthorne effect (ie, rates may have decreased because OR personnel knew they were being measured). The authors argued against this possibility based on two aspects of their data: this knowledge was in place before and after checklist introduction, and the subset of procedures for which study personnel were present in the OR had the same reductions in complications as procedures when study personnel were absent from the OR.
The simple existence of a formal pause or preoperative briefing (which could be done without a ‘checklist’). Such a pause is a necessary component of the checklist.
Increased uptake of safety technologies (eg, administering antibiotics in the OR rather than in preoperative wards). This change could be considered a byproduct of checklist introduction (ie, hospitals made more antibiotics directly available in the OR because of the presence of an antibiotics-related item on the checklist).
A broad change in safety culture and teamwork at that site, an explanation supported by the finding that greater increases in safety attitudes at the pilot sites were associated with greater reductions in complications.12
Subsequent publications about the WHO Surgical Safety Checklist have found improvements in urgent surgery13 and safety attitudes.12 ,14 Haynes et al12 reported that 80% of respondents considered the checklist easy to use, 20% believed it took too long and 93% of respondents would want the checklist used if they were undergoing surgery. Likewise, Helmio and colleagues15 found that 76% of OR staff agreed the checklist improved safety, 68% agreed it improved error prevention and 93% would want the checklist used if they were having surgery. Team members reported high satisfaction and positivity about the checklist, and estimated that it only took about 2 min to complete.16
The WHO checklist focuses primarily on events occurring within the OR. However, an estimated 53–70% of surgical errors occur outside the OR.8 ,17 ,18 The SURPASS checklist7–10 attempts to address these errors by encompassing all care between patient admission and discharge. Within the OR itself, the SURPASS checklist is less specific than the WHO checklist (eg, the SURPASS checklist does not specifically mention any of the following: pulse oximetry, difficult airway, risk of blood loss (although it asks whether blood products are available), team introductions, and anticipation of critical events).
De Vries et al7 tested the 90-item SURPASS checklist. In six test hospitals, the 3-month period after the checklist was initiated (compared with the 3 months before) saw numerous improvements: decreases in the percentage of patients with complications, in-hospital mortality, patient temporary disability and reoperations. No such improvements were found among the five control hospitals. Interestingly, the degree of improvement was associated with greater compliance with the checklist, providing greater confidence that the checklist itself was responsible for improvements. A subsequent retrospective review of 294 medical claims10 estimated that 40% of deaths and 29% of liability incidents might have been prevented if the SURPASS checklist had been used. Further review of 6313 checklists performed found that 41% detected at least one oversight, with the most common occurring postoperatively (lack of postoperative instructions concerning ventilation by the anaesthesiologist and missing medication prescriptions at discharge).19
Wrong-site surgery checklists
In January 2004, the Joint Commission launched the first version of the UP for Preventing Wrong Site, Wrong Procedure, Wrong Person Surgery.6 ,20 Preoperative verifications of person, procedure and site are supposed to occur in the OR and (if applicable) when the procedure is scheduled, when the patient enters the healthcare facility, and anytime care is transferred between caregivers. Site marking should involve only the operative site and should be visible before the patient is draped. The ‘time out’ is to occur before incision and involve the entire OR team. The UP is not a checklist21 but could be implemented using one or more checklists. Steps 1 and 3 specifically mention the potential use of a checklist.
Wrong-site surgery is rare; estimates for various procedures range from 1 in 13 000 procedures for wrong-site anaesthesia block to 1 in 4200 for wrong-side ureteral stents.22 A general systematic review estimated that the overall rate was 1–5 per 10 000 procedures.23 Given the rarity, demonstrating a statistical reduction would require an unfeasibly large study. A systematic review searched for literature and concluded there was ‘no literature to substantiate the effectiveness of the current Joint Commission Universal Protocol in decreasing the rate of wrong site, wrong level surgery.’23 Therefore, the preventive benefits of a checklist to prevent wrong-site surgery are generally assumed based on clinical expertise.
Direct harms of surgical checklists have not been reported. In 2011, Sewell et al24 reported that after WHO implementation, the rate of lower respiratory tract infections actually increased from 2.1% to 2.5%. Whether this increase was caused by the checklist is unclear; however the authors attributed rate reductions to the checklist, so they could also have attributed rate increases to the checklist. Despite the absence of reported direct harms, some checklist users have expressed concern regarding potential harms. For instance, some worry that checklist use decreases OR efficiency or creates unnecessary patient anxiety. In 2011, Kearns et al25 reported that 3 months after WHO checklist implementation, 30% believed it was an inconvenience in emergency cases; however, this percentage was lower than it had been prior to implementation of the checklist when staff were asked hypothetically whether they believed it would be an inconvenience in emergency cases (53% said it would be). OR efficiency might also be compromised if checklists duplicated already existing safety procedures or if nurses responsible for performing the checklist were unfamiliar with its execution due to high staffing turnover. 26 ,27 In one study,27 staff expressed concerns that prompting patients for their name several times immediately before induction of anaesthesia might create unnecessary anxiety.
Implementation considerations and costs
We included 23 reports of WHO checklist implementation. Twenty-one studies reported WHO checklist implementation at other sites and two reported experience at institutions involved in the original study (table 2).
Results from the 23 implementation reports appear in table 3. In keeping with WHO recommendations, checklists were tailored and implemented differently for a wide variety of contexts. At present, it remains unclear whether OR posters, paper tick boxes or electronic medical records perform better. Feedback from surgical teams was generally positive, but support tended to be greater from nurses and anaesthetists than from surgeons. For example, Vats et al26 found that anaesthetists and nurses were ‘largely supportive’ but some surgeons were ‘not very enthusiastic’.
Reasons cited for success included good training and staff understanding, a local champion, support from upper management, being able to modify the checklist, distribution of responsibility, the feeling of ownership by team members, a stepwise implementation process which incorporated real-time feedback, and enhanced communication and teamwork. Regarding communication, for example, Sewell et al24 found that 77% of users thought the checklist improved team communication; this percentage was 70% in the study by Kearns et al.25 The implementation study by Conley et al28 emphasised that the local champion should ‘persuasively explain why and adaptively show how to use the checklist’. Styer et al29 and Bohmer et al30 attributed success to recruiting senior leaders of their institutions to be local champions and incorporating real-time feedback into checklist protocols.
Barriers to implementation generally fell into four categories: confusion regarding how to properly use the checklist, pragmatic challenges to efficient workflow, access to resources, and individual beliefs and attitudes. First, OR staff were sometimes confused about how to properly execute the checklist.15 ,27 ,31 For instance, Levy et al31 found significant confusion about the timing of checklist items and who was responsible for prompting checklist questions among OR staff. While inadequate education may play a part, Fourcade et al27 found that nurses were unfamiliar with the checklist because of high staffing turnover. Vogts et al32 suggested that performance of ‘sign out’ may be low since this section is not linked to a specific event in patient management, unlike the ‘sign in’ and ‘time out’ domains and thus lacks clarity.
Second, checklist implementation occasionally created pragmatic problems for OR workflow. Particular challenges include extra time,27 ,32 especially during emergency procedures,33 and duplication of safety checks already routinely performed.26 ,27 In the study by Kearns et al25 30% felt that in emergency cases, the checklist was inconvenient. Third, developing countries often lacked regular access to resources. Yuan et al14 reported that inconsistent access to antibiotics and batteries hampered checklist use in two Liberian hospitals. Likewise, Kasatpibal et al34 reported that surgical sites were not routinely marked because marking materials were unavailable in a Thai hospital. Finally, individual attitudes of staff towards the checklist played a major role in the outcome of implementation. Barriers included general surgeon resistance to changing habits, awkwardness of self-introductions and steep interpersonal hierarchy. Some nurses reported concerns about incurring legal responsibility if a complication occurred after they signed the checklist form.
In terms of improved health outcomes (rightmost columns of table 3), 10 of the 21 implementation studies reported relevant data. Among the 10 reporting studies, however, reductions were generally impressive. For example, Askarian et al35 found that surgical complications decreased from 22.9% to 10%. Yuan et al14 reported that two Liberian hospitals found checklist introduction was significantly associated with fewer surgical site infections (adjusted OR (AOR) 0.28; 95% CI 0.15 to 0.54) and surgical complications (AOR 0.45; 95% CI 0.26 to 0.78).
Education and compliance
Regarding checklist training, 10 sites mentioned educational sessions, seven used posters in the OR, two mentioned a hospital-wide publicity campaign, two mentioned that training was provided (however no details were given), and eight either failed to mention training or stated that only limited training was provided. Six studies mentioned a pilot testing period; these pilot tests lasted 1–3 months and often resulted in minor modifications to the checklist.
Nine studies reported the degree of compliance with the checklist; one simply reported 97% compliance, and two others reported improvement over time (from approximately 60% to 80% in one study, and from 85% to 95% in another study). Notably, while compliance with checklist use was high, the checklists were often left incomplete. Fourcade et al27 reported checklist use in 90.2% of surgeries, but completion in only 61%. Similarly, Levy et al31 found that although checklist compliance was 100% in the electronic medical record, only 4 of 172 checklists completed more than 7 out of 13 required checklist items. Kasatpibal et al34 reported that staff had high compliance with checklist items which had already been standard hospital policy, but low compliance for checklist items not routinely practiced.
Our searches identified no attempts to use the SURPASS checklist outside the Netherlands. The website (http://www.surpass-checklist.nl/home.jsf?lang=en) describes a web version of the checklist (called SURPASS Digital), which allows one to modify the checklist, although the designers of SURPASS strongly discourage it (http://www.surpass-checklist.nl/content.jsf?pageId=FAQ&lang=en).
Wrong-site surgery checklists
We identified four sites describing checklists based on the Joint Commission's UP (table 4). The Swiss study37 focused on verifying patient identity and surgical site. Compared with the first 3 months of implementation, the next 3 months saw better compliance in checking patient identity and proportion of surgical site checks performed. Barriers to implementation included surgeons saying they already knew the patients or the surgical site was obvious, and the failure to include the input of all surgical services in developing the protocol.
The Swedish study38 involved two hospitals, each of which had a recent wrong-site surgery incident, and a root-cause analysis suggested that a time-out procedure might help. A time-out checklist was implemented, and 1 year later, a questionnaire showed that 93% of team members believed the checklist contributed to patient safety.
The English study39 was conducted at a children's hospital in which staff had incorporated an eight-item correct-site surgery checklist into an existing surgical checklist. Comparing 2008 with 2006, correct completion was improved for four of the eight checklist items.
The North Carolina study37 implemented a checklist to prevent wrong-site surgery that was tailored to the hospital's preferences and procedures. Staff commented favourably that they no longer had to remember everything on a cumbersome form.
No implementation advice was found on the Joint Commission website or in other published documents. In August 2010, the Joint Commission conducted an online survey of over 2100 people.40 The website reports high agreement that organisations can fully implement the UP, its three steps are appropriate, and that ‘there is benefit’ in using it in the OR, ambulatory surgery and hospital units performing invasive procedures (but the rates of agreement of benefit were lower for ambulatory clinics and physician offices). The need to modify policies and procedures varied greatly across respondents, and no differences were found between different types of respondents (eg, type of hospital, bed size).
Costs of implementing a checklist mostly involve checklist development and/or modification, formal staff notification, training and additional OR time. In 2010, Semel et al41 performed a hypothetical decision analysis of checklist introduction. The cost was estimated using the ‘opportunity cost of the work that would have otherwise been performed by the three department checklist champions and the implementation coordinator’, which was an estimated $12 635 in 2008 dollars; per-use cost was only $11. But the cost of a major surgical complication was estimated at $13 372. In the base case, checklist introduction saved money.
Regarding time, Sewell et al24 reported that 20% of staff thought the WHO checklist caused an unnecessary time delay. However, in 2011, Taylor et al16 reported that the WHO checklist took only about 2 min on average.
Adoption and diffusion
On 15 May 2013, the WHO's Surgical Safety Web Map (http://maps.cga.harvard.edu:8080/Hospital/) indicated that as of 26 March 2012, 4132 hospitals had expressed interest in using the checklist and 1790 of these hospitals have used the checklist in at least one operating theatre.
Many professional organisations have recommended adoption of the WHO checklist. These include the Institute for Healthcare Improvement (http://www.ihi.org), the National Patient Safety Agency in the UK (http://www.nrls.npsa.nhs.uk),42 ,43 the Canadian Patient Safety Institute,44 ,45 the Washington State Surgical Care and Outcome Assessment Program46 (http://www.scoap.org), the South Carolina Hospital Association,44 the Spanish Ministry of Health and Spanish Association of Surgeons,47 and the countries of France,44 Ireland,48 Jordan48 and the Netherlands.33 Furthermore, several organisations in Australia and New Zealand have developed modified versions of the WHO checklist: the Royal Australasian College of Surgeons, the Australian and New Zealand College of Anaesthetists, the Royal Australian and New Zealand College of Obstetricians and Gynecologists, the Australian College of Operating Room Nurses, and the Australian Commission for Safety and Quality in Health Care.49
Sivathasan and colleagues50 conducted telephone interviews with 238 hospitals in the UK. Almost all (99%) of the hospitals had heard of the checklist, and its use was already compulsory in 65% of them. In hospitals where it was not required, 81% used it voluntarily and 75% planned to make it mandatory in the future. Notably, some ORs reported partial use of the checklist, that is, intentionally skipping items or skipping the entire checklist because of time constraints.
In June 2009, the journal OR Manager received online data from 136 subscribers regarding use of the WHO checklist.51 Nearly half (48.5%) reported implementing the checklist and 64% said the checklist had improved safety in the OR. However, 11% of respondents stated that the checklist was not well accepted by surgeons and another 63% said surgeons did accept it but ‘with reservations.’ Nurses were found to have a somewhat greater degree of acceptance.
A 2009 UK survey of 12 oral and maxillofacial consultants found that all were aware of the WHO checklist, but only 5 of 12 were actually using it.52 Ten of 12 expressed the belief that it would improve patient safety, but 4 of 12 said it would not improve team communication.
Regarding the UP, accredited hospitals are required to comply. Therefore the ‘diffusion’ of the UP is large, by mandate. However, as stated earlier, the UP is not a checklist. We found no published information on how many hospitals actually use a checklist in their efforts to comply.
Several prominent authorities in the field of patient safety have promoted checklists in an attempt to prevent mistakes related to surgery. Our report demonstrates that checklists have been widely adopted, not only in Western countries, but in diverse contexts throughout the world. Notably, we found evidence that checklists are associated with improved health outcomes, including decreased surgical complications and surgical site infections. Association, however, does not imply causation. Thus, we note three important caveats. First, checklists are often implemented as part of a multifaceted strategy to improve care, which may render it difficult to determine whether improvements should be attributed to checklists alone or to other changes such as improved communication and shifts in OR culture. Second, reporting bias may have played a role. Eleven out of 21 implementation studies did not report health outcomes, potentially due to an absence of clear improvements after checklist implementation. Third, the reported results do not mean that all surgical checklists are beneficial; other surgical checklists containing different items may or may not be beneficial.
Many surgical staff have reported favourable attitudes towards checklist implementation. However, numerous implementation issues remain, including how to modify a given checklist to a specific hospital setting or specific surgical staff. Our report found that barriers to effective implementation include confusion regarding practical aspects of checklist use, dealing with challenges to efficient workflow, obtaining regular access to resources and the beliefs and attitudes of participating staff, particularly surgeons. One recurrent theme in the literature on surgical checklists is the explicit encouragement of a team-based approach. The AHRQ continues to investigate factors supportive of effective checklist implementation with the 2010–2013 project entitled, ‘Factors associated with effective implementation of a surgical safety checklist’.53 This project will elucidate how teamwork may contribute to the impact of the checklist.
The WHO checklist's wide adoption and dissemination suggests it may serve as a model for policymakers seeking to develop safety strategies in the future. This checklist was explicitly designed to be modified for widely varying contexts and executed in a short time frame to maintain feasibility. The WHO website instructs hospitals: ‘Do not hesitate to customise the checklist for your setting as necessary, but do not remove safety steps just because you are unable to accomplish them’ and emphasises that ‘It should take no more than a minute to complete each section of the checklist’ (ie, 3 min in total).54 The pilot study reported that, at various sites, introduction of the checklist took only 1 week to 1 month.5 Checklist implementation is relatively inexpensive, with some hospitals simply printing posters to be hung on OR walls. These practical characteristics of the WHO checklist may have significantly promoted its uptake and use. Notably, the WHO approach markedly differs from that stated by creators of the SURPASS checklist, who strongly discouraged its adaption. Although SURPASS is more comprehensive, it has not been widely implemented, potentially due to the resource intensive effort required to track patients throughout a surgical hospitalisation.
In conclusion, the WHO checklist, the SURPASS checklist and checklists implementing the Joint Commission UP represent promising initiatives with suggestive evidence for improving patient safety. Future research may clarify the unique nature of their contribution and provide insights for effective implementation.
Key summary points
Surgical checklists such as the WHO Surgical Safety Checklist and Surgical Patient Safety System (SURPASS) checklist offer a promising intervention for decreasing patient morbidity and mortality due to surgical operations.
The WHO Surgical Safety Checklist has been successfully adapted for implementation in a wide variety of settings, including all surgical specialties, academic and community hospitals, and industrialised and developing countries.
Surgical safety checklists were associated with increased detection of potential safety hazards, decreased surgical complications and improved communication among operating room staff. Other factors independent of checklists, such as concurrent safety improvements, may also explain these improvements.
Key components of successful checklist implementation include enlisting support from institutional leaders, training staff on using the checklist, adapting the checklist to incorporate staff feedback and avoiding the duplication of information already routinely collected.
The authors wish to thank Allison Gross, MS, MLS for performing literature searches, and Karen Schoelles, MD, SM and Paul Shekelle, MD, MPH, PhD for project oversight.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.