Article Text

PDF
A behaviourally anchored rating scale for evaluating the use of the WHO surgical safety checklist: development and initial evaluation of the WHOBARS
  1. Daniel A Devcich1,
  2. Jennifer Weller1,2,
  3. Simon J Mitchell1,2,
  4. Scott McLaughlin3,
  5. Lauren Barker3,
  6. Jenny W Rudolph4,
  7. Daniel B Raemer4,
  8. Martin Zammert5,
  9. Sara J Singer6,
  10. Jane Torrie1,2,
  11. Chris MA Frampton7,
  12. Alan F Merry1,2
  1. 1Department of Anaesthesiology, University of Auckland, Auckland, New Zealand
  2. 2Department of Anaesthesia and Perioperative Medicine, Auckland City Hospital, Auckland, New Zealand
  3. 3School of Medicine, University of Auckland, Auckland, New Zealand
  4. 4Harvard Medical School, Center for Medical Simulation, Boston, Massachusetts, United States
  5. 5Harvard Medical School, Brigham and Women's Hospital, Boston, Massachusetts, United States
  6. 6Department of Health Policy and Management, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, United States
  7. 7Department of Medicine, Christchurch School of Medicine and Health Sciences, University of Otago, Auckland, New Zealand
  1. Correspondence to Professor Alan F Merry, Department of Anaesthesiology, University of Auckland, Auckland New Zealand a.merry{at}auckland.ac.nz

Abstract

Background Realising the full potential of the WHO Surgical Safety Checklist (SSC) to reduce perioperative harm requires the constructive engagement of all operating room (OR) team members during its administration. To facilitate research on SSC implementation, a valid and reliable instrument is needed for measuring OR team behaviours during its administration. We developed a behaviourally anchored rating scale (BARS) for this purpose.

Methods We used a modified Delphi process, involving 16 subject matter experts, to compile a BARS with behavioural domains applicable to all three phases of the SSC. We evaluated the instrument in 80 adult OR cases and 30 simulated cases using two medical student raters and seven expert raters, respectively. Intraclass correlation coefficients were calculated to assess inter-rater reliability. Internal consistency and instrument discrimination were explored. Sample size estimates for potential study designs using the instrument were calculated.

Results The Delphi process resulted in a BARS instrument (the WHOBARS) with five behavioural domains. Intraclass correlation coefficients calculated from the OR cases exceeded 0.80 for 80% of the instrument's domains across the SSC phases. The WHOBARS showed high internal consistency across the three phases of the SSC and ability to discriminate among surgical cases in both clinical and simulated settings. Fewer than 20 cases per group would be required to show a difference of 1 point between groups in studies of the SSC, where α=0.05 and β=0.8.

Conclusion We have developed a generic instrument for comprehensively rating the administration of the SSC and informing initiatives to realise its full potential. We have provided data supporting its capacity for discrimination, internal consistency and inter-rater reliability. Further psychometric evaluation is warranted.

Statistics from Altmetric.com

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.