Article Text

Download PDFPDF
Determination of the psychometric properties of a behavioural marking system for obstetrical team training using high-fidelity simulation
  1. Pamela J Morgan1,
  2. Deborah Tregunno2,
  3. Richard Pittini3,
  4. Jordan Tarshis4,
  5. Glenn Regehr5,
  6. Susan Desousa4,
  7. Matt Kurrek6,
  8. Ken Milne7
  1. 1Department of Anesthesia, Women's College Hospital and Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
  2. 2School of Nursing, York University, Toronto, Ontario, Canada
  3. 3Department of Obstetrics and Gynecology, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
  4. 4Department of Anesthesia, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada
  5. 5Centre for Health Education Scholarship, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
  6. 6Department of Anesthesia, University of Toronto, Toronto, Ontario, Canada
  7. 7Salus Global Corporation, London, Ontario, Canada
  1. Correspondence to Dr Pamela J Morgan, Department of Anesthesia, Women's College Hospital, 76 Grenville Street, Toronto, ON M5S 1B2, Canada; pam.morgan{at}


Background To determine the effectiveness of high-fidelity simulation for team training, a valid and reliable tool is required. This study investigated the internal consistency, inter-rater reliability and test–retest reliability of two newly developed tools to assess obstetrical team performance.

Methods After research ethics board approval, multidisciplinary obstetrical teams participated in three sessions separated by 5–9 months and managed four high-fidelity simulation scenarios. Two tools, an 18-item Assessment of Obstetric Team Performance (AOTP) and a six-item Global Assessment of Obstetric Team Performance (GAOTP) were used.5 Eight reviewers rated the DVDs of all teams' performances.

Results Two AOTP items were consistently incomplete and omitted from the analyses. Cronbach's α for the 16-item AOTP was 0.96, and 0.91 for the six-item GAOTP. The eight-rater α for the GAOTP was 0.81 (single-rater intra-class correlation coefficient, 0.34) indicating acceptable inter-rater reliability. The ‘four-scenario’ α for the 12 teams was 0.79 for session 1, 0.88 for session 2, and 0.86 for session 3, suggesting that performance is not being strongly affected by the context specificity of the cases. Pearson's correlation of team performance scores for the four scenarios were 0.59, 0.35, 0.40 and 0.33, and for the total score across scenarios it was 0.47, indicating moderate test–retest reliability.

Conclusions The results from this study indicate that the GAOTP would be a sufficient assessment tool for obstetrical team performance using simulation provided that it is used to assess teams with at least eight raters to ensure a sufficiently stable score. This could allow the quantitative evaluation of an educational intervention.

  • Education continuing
  • obstetric
  • safety
  • health professions education
  • communication
  • continuing education
  • continuing professional development
  • crew resource management
  • patient safety
  • safety culture
  • healthcare quality improvement
  • simulation
  • crisis management
  • evaluation methodology
  • human factors
  • obstetrics and gynecology

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • Funding This work was supported by a research grant from the Canadian Patient Safety Institute (CPSI), Edmonton, Alberta, Canada. Research equipment support was received from Hedstrom Canada, Cambridge, Ontario, Canada.

  • Competing interests Drs Morgan, Tregunno, Pittini, Regehr, Kurrek and Ms DeSousa have no conflict of interest related to this study. Dr Tarshis is a shareholder in CAE Inc. and Dr Ken Milne is a salaried employee in the position of President and CEO of Salus Global Corporation.

  • Ethics approval Ethics approval was provided by Sunnybrook Health Sciences Centre.

  • Provenance and peer review Not commissioned; externally peer reviewed.