Monitoring universal protocol compliance through real-time clandestine observation by medical students results in performance improvement

Objective: To measure universal protocol compliance through real-time, clandestine observation by medical students compared with chart audit reviews, and to enable medical students the opportunity to become conscious of the importance of medical errors and safety initiatives.

Design: With endorsement from Tufts Medical Center's (TMC's) Chief Medical Officer and Surgeon-in-Chief, 8 medical students performed clandestine observation audits of 98 cases from April to August 2009. A compliance checklist was based on TMC's presurgical checklist. Our initial results led to interventions to improve our universal protocol procedures, including modifications to the operating room white board and presurgical checklist, and specific feedback to surgical departments. One year later, 6 medical students performed observations of 100 cases from June to August 2010.

Setting: Tufts Medical Center, Boston, Massachusetts, which is an academic medical center and the principal teaching hospital for Tufts University School of Medicine.

Participants: An operating room coordinator placed the medical students into 1 of our 25 operating rooms with students entering under the premise of observing the anesthesiologist for clinical education. The observations were performed Monday to Friday between 7 am and 4 pm. Although observations were not randomized, no single service or type of surgery was targeted for observation.

Results: A broad range of departments was observed. In 8.2% of cases, the surgical site was unmarked. A Time Out occurred in 89.7% of cases. The entire surgical team was attentive during the time out in 82% of cases. The presurgical checklist was incomplete before incision in 13 cases. Images were displayed in 82% of cases. The operating room "white board" was filled out completely in 49% of cases. Team introductions occurred in 13 cases. One year later, compliance increased in all Universal Protocol dimensions.

Conclusions: Direct, real-time observation by medical students provides an accurate and granular assessment of compliance with specific components of the universal protocol and engages medical students in the quality improvement process, raises their awareness of the gravity of medical errors, and ensures appreciation of the importance of quality and safety initiatives.