A large number of adverse events (AEs) are encountered during hospital admission and in particular around surgical procedures.1^–5 The operating room is a complex environment with a number of factors that enhance the risk of AEs occurring. It contains a high concentration of information and patient transfers, and many man–machine interactions.67 This complexity is reflected in the fact that most AE studies show that a large proportion of the events are related to an operation.58

One of the recommendations made by the Institute of Medicine in its well-known 1999 report “To err is human” was the implementation of verification processes such as checklists into medical practice.9 Checklists increase standardisation in work processes and avoid reliance on memory, decreasing the chances of human error. Safety checks targeted to known error risks have been shown to be effective in other areas where safety hazards are encountered, such as aviation and high-risk industries. Whereas domestic aviation has achieved a fourfold increase in safety in a 15-year period and Occupational Health and Safety Administration programmes have reduced the workplace death rate by half in a similar period of time, the medical community in general lags behind in this matter.1011 Although dramatic decreases in anaesthesia mortalities have been reported,9 this claim has been shown to be misleading.12

As the probability of medical errors in surgical patients appears to be influenced by factors similar to aviation, comparable procedural safety checks may be as effective in surgery as they are in aviation to prevent errors.1314 The first step in testing this hypothesis is to construct a surgical procedural safety checklist along the lines of the airline industry (pretakeoff) checks. Thereafter, effectiveness can be evaluated. In anaesthesiology, the use of checklists and protocols was shown to be associated with a decreased risk of perioperative mortality.15 In recent years, a number of groups have investigated and implemented surgical briefing checklists to be used during a preoperative time-out.1617 However, the use of checklists in surgery, in particular in the complete in-hospital surgical pathway including admission and discharge, is as yet not widespread, and validated checklists for the entire surgical pathway do not exist.13 The aim of this study was to develop and validate a “surgical patient safety system” (SURPASS) checklist that accompanies the patient admitted for surgery until discharge and to evaluate its use in clinical practice.

METHODS

We aimed to create a checklist that covered the maximum of relevant safety risk events. We did this using a three-stage strategy:

1. the collection of all known surgical safety risk events from the relevant literature (development of prototype checklist);

2. checking whether these theoretical safety risk events matched the safety risk events occurring in practice (validation);

3. the evaluation of the clinical use of the instrument.

Development of prototype checklist

Human-factors literature, originating mainly from aviation experience, was consulted for information on the ideal format, design, length and graphic layout of a checklist.18^–20

To determine the contents of the list, a large number of sources concerning the causes, nature and locations of surgical errors, complications and adverse events were consulted.821^–32 The surgical pathway was then divided into distinct stages and the critical safety risks in each of these stages were identified and formulated as items on the different parts of the checklist. The information derived from both literature studies was combined to form the prototype SURPASS checklist.

Item validation by observation

Checklists are only potentially useful and effective when the items on the list match the safety risk events that actually occur in practice.20 Consequently, we set out to validate the items on the checklist by real-time observation of the surgical pathway and evaluation of the extent of agreement between process deviations occurring in reality and items on our theory-based checklist. All of these deviations from the optimal care process are safety-risk events and have the potential for adverse outcomes. However, the consequences of the process deviations were not part of this study; thus, registered deviations did not necessarily lead to adverse outcomes.

In a first validation period, 41 surgical procedures were arbitrarily selected from the elective operating schedule in a tertiary referral centre (Academic Medical Centre in Amsterdam, The Netherlands) (table 1). These procedures were observed by an independent researcher, and a minute-by-minute narrative report was made of all activities. Each report was analysed by an expert panel, consisting of a surgeon and two safety experts, to extract the moments where deviations from the usual and/or optimal process occurred. The safety expert panel compared each procedure report with existing protocols and usual standards of care, and judged whether or not the observed deviations were appropriate for that particular situation; only those process deviations that were judged “inappropriate” or “undesirable” were considered.

Based on this first set of observation data and other sources such as the Australian Incident Monitoring System Taxonomy,33 an observation form was constructed. Unlike the (prototype) checklist (created using existing literature), this taxonomy observation form was based on observed process deviations. The form was used to observe the second set of procedures in a more structured and categorised manner. The observation form was non-exclusive; all process deviations were registered.

In the second validation period, another 130 surgical procedures were observed in two phases (30 and 100 observations, respectively; table 1). These procedures were selected based on the frequency of procedures multiplied by the percentage of complications, using the hospital’s surgical complication registration, and termed “sentinel” procedures (table 2). Together, these procedures represent two-thirds of all procedures and over 90% of all complications in a tertiary referral centre. In addition to process observations, complementary questionnaires were filled out by all personnel in the OR and on the ward at several moments for each observed sentinel procedure. Again, process devations were analysed by an expert panel.

Subsequently, the observed process deviations from all 171 procedures were compared with the prototype (theory-based) checklist and scored as: “corresponding to item on checklist,” “not corresponding to item on checklist” and “not suitable for checklist.” Using these data, the checklist was adapted.

Evaluation

After the validation period, the checklist was introduced in daily practice, after instructive presentations for all users, and accompanied all elective procedures from the aforementioned sentinel categories. The checklist was affixed to the cover of the patient’s case notes. During the first 2 months of checklist use, a researcher present in the operating room and on the wards actively encouraged personnel to fill out the list. However, during this evaluation period, no consequences were attached to not completing the list: the process continued whether or not the list was filled out.

After 5 months of checklist use, a structured interview was performed with staff and residents who had been involved in checklist use in practice.

RESULTS

Development of checklist

The SURPASS checklist accompanies the patient during each step of the surgical pathway (fig 1). It is split up into different stages (preoperative ward, operating room, recovery or intensive care unit, postoperative ward) and focuses on the patient transfer moments in between (including admission and discharge). The list is multidisciplinary: ward doctor, surgeon, anaesthesiologist, OR-assistant and nurse are all responsible for completion of parts of the checklist.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Structure of the optimised Surgical Patient Safety System checklist in relation to surgical pathway. ICU, intensive care unit.

Item validation by observation

In two validation periods, a total of 171 procedures were observed. During these procedures, a total of 593 process deviations were noted (tables 3, 4). In the preoperative phase, 221 deviations were observed, 98% of which corresponded to an item on the SURPASS checklist. Most of these deviations were related to informed consent and medication orders. During surgery, 250 deviations were observed, 44% of which corresponded to an item on the checklist. Deviations that corresponded to the checklist included peroperative medication, information and the presence of instruments or material. Of deviations observed during surgery, 49% were considered unsuitable for a checklist, mostly related to malfunctioning of instruments in sterile nets and personnel logistics. In the postoperative phase, 122 deviations were observed, 93% of which corresponded to an item on the checklist. This concerned mostly missing information and insufficient postoperative instructions.

When the deviations unsuitable for a checklist item were not considered, 96% of observed deviations corresponded to an item on the SURPASS checklist.

Evaluation

Before starting the evaluation, the SURPASS checklist was adapted based on the process observation data. Some items were added or rephrased. Subsequently, the adapted checklists ran concurrent with all sentinel procedures (350) performed during a 5-month period. After this period of checklist use, extensive structured interviews for evaluation and discussion were held with 21 surgeons and surgical residents, and 17 anaesthesiologists. Ninety-five per cent of interviewees reported to have completed the checklist “mostly” or “always” (table 5). Different motivations were mentioned for not filling out the checklist (table 5). Lack of time was mentioned by 34% of interviewees; 66% forgot about the checklist now and then. When asked to give suggestions to improve logistics and content of the list, 45% of interviewed doctors suggested integrating the checklist into existing hospital information systems, while 13% felt that compliance would increase if consequences (stopping rules) would be attached to checklist use (table 5). Members of the nursing staff had few remarks and were generally content with the checklist.

Final adaptation

In the final version of the SURPASS checklist, comments and suggestions of all interviewed personnel were incorporated. In addition, to enhance communication about the presurgery checks, the separate OR checklist pages for the surgeon and the anaesthesiologist were merged into one preoperative time-out form, prior to the start of the procedure (part B of the checklist, fig 1). This procedure consists of a short preoperative discussion between surgeon, anaesthesiologist and OR-assistant.

DISCUSSION

Based on the best available knowledge regarding the nature of safety risks in surgery and regarding checklist design, we constructed a prototype multidisciplinary SURPASS checklist accompanying the patient along the entire surgical pathway.

Of the process deviations or safety-risk events that by their nature are suitable for coverage by a checklist, virtually all corresponded to an item on the SURPASS list. About half of the process deviations observed during surgery did not correspond to a checklist item but were deemed unsuitable for coverage by a checklist. Deviations concerning personnel and logistics are evidently unsuitable for a checklist by nature of their unpredictability. For example, it is impossible to check in advance that no member of the surgical team will be delayed unexpectedly. Likewise, if sterile equipment, when unwrapped, is shown to malfunction, this could not have been checked in advance by a patient-oriented checklist. However, while the SURPASS checklist cannot prevent these problems directly, the inclusion of a time-out procedure with a multidisciplinary preoperative discussion might lead, for example, to fewer problems with delayed personnel. In addition, if checklist use reveals recurrent problems, for example with sterile equipment, this could lead to structural measures such as more stringent checks of instruments or equipment before sterilisation.

When evaluating the checklist, most interviewees were generally positive. Important reasons for non-compliance were the absence of consequences when the checklist was not completed and the fact that the checklist was not integrated into existing hospital information systems. These aspects are partly related to attitude and safety culture but could be solved, at least in part, by integrating the checklist into an electronic system. An electronic version of the SURPASS checklist is presently being developed, which will then need integration into existing hospital information systems to be linked to patient data and protocols.

A possible limitation of this study is that the observations were influenced by the fact that surgeons, anaesthesiologists and operating assistants knew that they were being observed. This might have led to an underestimation of the number of deviations, because of increased diligence to ensure that everything went according to protocol. However, the large number of observed high-safety-risk procedures yielded a considerable number of deviations and should allow for a reliable cross-section.

Observation techniques have been used before to evaluate patient safety in the operating room. Christian et al observed 10 complex surgical cases, performing a detailed analysis of all observed events, and predominantly found problems in communication and information flow, and workload and competing tasks.6 Other studies have looked explicitly at interpersonal factors or at communication problems.3435 In the present study, we did not perform an in-depth analysis of the observed process deviations, but rather tried to gather an overview of the variety of deviations in a larger number of cases.

A number of studies have been conducted that evaluated the use of checklists in medicine.16173637 An aviation-style checklist has been evaluated in anaesthesia and was generally considered useful, although its applicability in clinical practice was doubtful.36 Recently, a briefing tool was described that consists of a preoperative time-out procedure guided by a short checklist.17 The SURPASS checklist includes a preoperative time-out similar to this briefing tool, but in addition the present checklist covers the entire surgical pathway from admission to discharge, instead of focusing merely on the immediately preoperative phase.

The SURPASS checklist has not yet been proven to actually decrease the number of process deviations or adverse events in the surgical pathway. Before widespread implementation of this instrument can be advised, effectiveness needs to be demonstrated to ensure that the checklist is not merely an extra layer of administration, but actually contributes to patient safety. However, the checklist items did cover the vast majority of almost 600 process deviations in the surgical pathway, in particular in the pre- and postoperative phase, and SURPASS was tested in daily clinical practice. Medical or surgical checklists with comparably thorough methodological design are scarce, and none of the existing checklists covers the entire patient pathway from admission to discharge. In the current climate where patient safety is a very important issue, and the call for solutions is urgent, the SURPASS checklist is a relatively simple intervention that standardises the surgical pathway and makes it less dependent on human memory. The checklist was developed in a tertiary referral centre but validated in a diversity of surgical procedures, making it just as easily applicable in hospitals with a different patient profile. Currently, the effectiveness of the SURPASS checklist with respect to reduction of process deviations and adverse events and improvement of patient safety is being evaluated in various hospital settings. This effectiveness and implementation study is carried out in cooperation with four teaching hospitals; results are expected in 2009.

In conclusion, we developed, validated and evaluated a surgical patient safety checklist. The SURPASS checklist covered virtually all process deviations suitable for a checklist and was easy to use in clinical practice. It is the first validated patient safety checklist for the surgical pathway from admission to discharge.