Elsevier

Journal of Surgical Education

Volume 69, Issue 1, January–February 2012, Pages 41-46
Journal of Surgical Education

Original report
Monitoring Universal Protocol Compliance Through Real-Time Clandestine Observation by Medical Students Results in Performance Improvement

https://doi.org/10.1016/j.jsurg.2011.05.015Get rights and content

Objective

To measure universal protocol compliance through real-time, clandestine observation by medical students compared with chart audit reviews, and to enable medical students the opportunity to become conscious of the importance of medical errors and safety initiatives.

Design

With endorsement from Tufts Medical Center's (TMC's) Chief Medical Officer and Surgeon-in-Chief, 8 medical students performed clandestine observation audits of 98 cases from April to August 2009. A compliance checklist was based on TMC's presurgical checklist. Our initial results led to interventions to improve our universal protocol procedures, including modifications to the operating room white board and presurgical checklist, and specific feedback to surgical departments. One year later, 6 medical students performed observations of 100 cases from June to August 2010.

Setting

Tufts Medical Center, Boston, Massachusetts, which is an academic medical center and the principal teaching hospital for Tufts University School of Medicine.

Participants

An operating room coordinator placed the medical students into 1 of our 25 operating rooms with students entering under the premise of observing the anesthesiologist for clinical education. The observations were performed Monday to Friday between 7 am and 4 pm. Although observations were not randomized, no single service or type of surgery was targeted for observation.

Results

A broad range of departments was observed. In 8.2% of cases, the surgical site was unmarked. A Time Out occurred in 89.7% of cases. The entire surgical team was attentive during the time out in 82% of cases. The presurgical checklist was incomplete before incision in 13 cases. Images were displayed in 82% of cases. The operating room “white board” was filled out completely in 49% of cases. Team introductions occurred in 13 cases. One year later, compliance increased in all Universal Protocol dimensions.

Conclusions

Direct, real-time observation by medical students provides an accurate and granular assessment of compliance with specific components of the universal protocol and engages medical students in the quality improvement process, raises their awareness of the gravity of medical errors, and ensures appreciation of the importance of quality and safety initiatives.

Introduction

Although deemed a “never event” by Centers for Medicare and Medicaid Services, the Joint Commission ranks wrong-site surgery (WSS) as the most frequently reported sentinel event with 666 instances, accounting for 13.7% of 4850 events between 2004 to 2010.1 The Joint Commission issued Sentinel Event Alert newsletters regarding WSS in 1998 and 2001. The Universal Protocol aims to address the continuing occurrence of wrong-site, wrong-procedure, and wrong-person surgery in organizations accredited by the Joint Commission. It is composed of 3 principal components: conducting a preprocedure verification process, marking the procedure site, and performing a time out before the procedure. In January 2009, the Universal Protocol was updated based on 2007 WSS Summit feedback.

Thirteen percent of reported sentinel events were because of WSS. Factors that increase risk of WSS include emergency operations, unusual patient characteristics (physical deformity or massive obesity), special equipment setup or patient positioning, unusual time pressures, more than 1 surgeon involved in the case, or multiple procedures conducted on the same patient.2 WSS is more common in certain specialties, particularly those with surgeries involving laterality. Absence of radiographic images and wrong site labeling on the images also play a causative role in wrong site orthopedic and spinal procedures.3 Root causes identified by the hospitals usually involved more than 1 factor; however, most involved a breakdown in communication between surgical team members and the patient and family.2 Organizational culture, interpersonal dynamics, and steep hierarchal structures contribute to errors by creating an environment where persons are reluctant to speak up.4

Events such as wrong-site operations are too rare to measure as rates.5 Instead, hospital administrators often measure safety and quality protocol compliance through peer audits, self-report questionnaires, or chart audits.6, 7, 8, 9 Questionnaires and chart audits supply information conveniently, but these strategies alone may not be fully representative of universal protocol compliance and may not capture select components of the Joint Commission guidelines. For example, individual features of a hospital's universal protocol process may not be detailed in the chart, including correct incision site marking, display of appropriate radiographic images, discussion of pertinent patient care issues before incision, or “white board” filled out to completion. Compliance may simply be documented as whether or not the surgical time out was performed before incision. It is important to obtain data from the unique components of the universal protocol, as errors may be caused by an accumulation of minor mistakes. Direct observation of components of universal protocol compliance has been reported,10, 11, 12 but it can be labor and resource intensive and may be subject to bias because of oversampling selected shifts.7

At Tufts Medical Center, a WSS occurred in 2008. Operating room administration was challenged to develop a better method to monitor compliance, supply feedback to surgical teams, and to measure their impact on our safety related policies and procedures over time. Although our compliance was consistently reported at 100% per chart audit, we felt that we needed a more sensitive means of auditing compliance and decided to employ a clandestine direct surveillance approach with medical student observers to assess Universal Protocol compliance in real time. Furthermore, we believe these observations would provide education in medical errors to these students.

Section snippets

Methods

Eight medical students performed clandestine direct observation audits of Universal Protocol compliance of 98 cases during the period of April 2009 to August 2009. A compliance checklist, which was similar to surveys used by other institutions to monitor time out compliance, was created for data collection purposes based on Tufts Medical Center's presurgical checklist called the “ticket to safety” (TTS). In addition to compliance data, the observing student collected the names of the surgical

Results

Students assessed compliance with universal protocol procedures in a broad range of surgical services (Table 1). All findings (Table 2) were reviewed by the Chief Medical Officer and Surgeon-in-Chief of Tufts Medical Center. In some instances, the medical student was not able to ascertain item compliance without risking exposure of their clandestine purpose for being in the operating room. In those instances, item compliance was marked “unknown.”

On the initial survey, in 8.2% of cases, a

Discussion

Measurement and feedback are crucial to any quality improvement initiative. Currently, no standard system is in place to measure compliance of the universal protocol process. Process compliance is measured by outcomes (near misses and wrong site procedures), often when it is too late. Direct observation is the gold standard in compliance measurement and has been used successfully by other institutions to improve the quality of patient care.6 Observation as a strategy to measure compliance

Conclusions

Direct real-time observation provides an accurate and granular assessment of compliance with specific components of the universal protocol and a better understanding of where processes fail. In our experience, clandestine direct observation provided reliable compliance data, which in turn led to improved performance. Routine post hoc chart audits are insensitive in determining compliance, especially in time-sensitive processes, such as checklist completion. Furthermore, as conducted at our

Acknowledgments

The following individuals are medical students deserving of acknowledgment for their participation in operating room observations: Patrick G. Dermarkar, Aaron G. Edelstein, Sumeet V. Jain, Neal A. Biddick, Ryan A. Chandhoke, and Elizabeth H. Au at Tufts Medical Center.

References (13)

There are more references available in the full text version of this article.

Cited by (0)

View full text