Objective Cardiac surgery (PCS) has a low error tolerance, is dependent upon sophisticated organisational structures and demands high levels of cognitive and technical performance. The aim of the study was to assess the role of intraoperative non-routine events (NREs) and team performance on paediatric cardiac surgery outcomes. The current paper focuses on improving methods for studying teamwork; a companion paper will report on the empirical results.

Methods The authors trained human factors observers to observe and code the NRE's and teamwork from time of arrival of the patient into the operating room (OR) to the patient handover in the intensive care unit. The observers underwent immersive training in which each observer attended 10 operations, learnt in detail about the technical procedures and clinical tasks and received practice in coding teamwork. Two observers were used interchangeably to observe OR teamwork. The authors instigated a rigorous training and assessment protocol, with independent assessment of their performance by both senior medical and human factors experts using video-based assessment. Real-time teamwork observations were supplemented with process mapping, questionnaires on safety culture, level of preparedness by the team, difficulty of the operation and outcome measures.

Results 19 PCS cases were observed. The observers observed a total of 255 hr of operations, including the first 10 training cases. We found that 68% of events were observed by one observer but only 32% of all events were observed by both observers. If an event was coded by both observers, 76% was coded in the same way, resulting in high levels of inter-rater agreement. The inter rater reliability of the four main teamwork categories was 91% with Cohen kappa of 0.77. Recommendations were developed for observing teamwork in the operating room, for instance ‘train observers on video recordings of real operations (not scripted performance), preferably of at least 1–2 h in duration’ and ‘Rate teamwork in real time and not afterwards.’

Conclusions PCS is an ideal model to explore team performance. A challenge for the future is to make observations of teamwork in healthcare settings more efficient and robust.