We studied personnel performing laparoscopic cholecystectomy (LC) and carotid endarterectomy (CEA) procedures in the theatre suites of a major UK teaching hospital. These procedures were selected to minimise difficulties in evaluating performance caused by variations in the operative procedure. All CEA operations were performed in one theatre while the LC procedures were mostly done in three others, one of which was designated a day-case theatre. All CEA operations were elective or semielective procedures performed or directly supervised by consultant vascular surgeons. About 20% of LC were performed by experienced trainees without supervision, and a similar percentage were “hot” procedures. There were no major differences in availability and quality of equipment between the LC theatres except that intraoperative cholangiography was difficult to perform in the day-case theatre. The consultant surgical and anaesthetic staff and senior theatre nurses were largely constant throughout the study, but junior staff turnover was relatively high, typical of UK practice. The day-case theatre was situated in a different hospital from the other theatres; the nursing staff in this theatre were predominantly trained in the Philippines, whereas most staff in the other theatres were UK-trained. The core CEA team had considerable experience of working together, and the senior members had all been trained by the same senior vascular surgeon when the CEA operation was first practised regularly at the hospital. We aimed to study consecutive cases but made exceptions due to refusal of consent (one case) or unavailability of research staff (three cases). After the intervention, we also excluded cases where there were fewer than two members of the theatre team who had attended the classroom part of the training course (six cases).

PROBLEM

Current evidence suggests that modern healthcare causes unintentional harm to between 3% and 16% of hospital inpatients.12 The factors implicated include organisational complexity, reliance on potentially dangerous high-technology equipment and the lack of systematic communications and teamwork training for staff.34 The operating theatre is reportedly the environment where patient harm is most likely.5 Theatre work is perceived as stressful by staff,6 and communication between team members as flawed and difficult.7 Reports on high-profile disasters in surgery such as wrong organ removal or the Bristol cardiac surgery incident have stressed the major role of teamwork and communication problems in bringing these about.8 Reviews of practice in high-reliability industries such as aviation, nuclear power and offshore oil production emphasise the need for training in principles of teamwork and free communication to reduce the risks of error,9 and studies in healthcare show that such training improves safety culture and attitudes across a range of specialty settings.1011 We were aware that, like most large hospitals, our local Trust had experienced major intraoperative incidents such a wrong organ removal in the recent past. Observational studies in paediatric cardiac surgery have shown (a) a clear association between the accumulation of minor errors in an operation and the occurrence of a major incident with potentially serious implications for the patient12 and (b) a convincing correlation between the quality of theatre team “non-technical skills” and the number of technical errors which occurred.13 The environment in which these studies were conducted is highly unusual, and in particular combines extremely high risks of adverse or fatal outcome with an absolute need for close cooperation between anaesthetists, surgeons and perfusionists in the heart–lung bypass process. A study of non-technical skills and technical errors in LC in our theatre suite, however, showed evidence of deficiencies in both technical and non-technical skills of the same order as that seen in the cardiac environment, and confirmed the association between them.14 The hypothesis that improvements in theatre team non-tech skills could substantially reduce errors and adverse outcomes of surgery has become widely accepted, but the evidence supporting this hypothesis is currently scanty.

Studies of theatre teamwork, communication and safety have been largely observational, focusing on the development of methodology15^–17 and on the correlations between non-technical skills and adverse events.131819 Only one major interventional study has attempted to evaluate non-technical skills training in hospitals,20 reporting a reduction in clinical errors in emergency rooms. No similar study of theatre teams has been reported. We felt that the postulated benefits of non-tech skills training for theatre teams merited further study. Based on our reading, we expected we might, through aviation-style “CRM” teamwork training, be able to improve teamwork in theatre teams, and thereby reduce technical errors and misunderstandings. We therefore hoped to see reductions in the number of potentially significant errors and mishaps in theatre with potential for harm to patients, and perhaps to observe improvements in some clinical outcome measures. We therefore undertook a single-institution before–after study of a non-technical skills team training intervention.

KEY MEASURES FOR IMPROVEMENT

We first observed performance and outcomes in a series of operations, then introduced a training intervention, and finally conducted a second observation period. The three phases lasted 6, 3 and 6 months respectively. We initially agreed to observe a theatre team from another specialty without providing an intervention, but this team withdrew after some initial observations due to sensitivities over their perception that involvement called into question their current performance. The study was therefore an uncontrolled before–after intervention study. Our hypothesis postulated an effect of teamwork training starting with improved attitudes, knowledge and performance in terms of teamwork and communication, leading to a reduction in “unsalvaged” technical errors, misunderstandings and omissions, and via this route to less imperfect technical results and ultimately less complications. We therefore designed our outcome measures to evaluate a “chain of evidence” linking teamwork training via improved teamwork attitudes, skills and performance to reduced technical errors, untoward events and adverse outcomes. We therefore measured attitudes, knowledge and behaviour relating to workplace teamwork and safety, and recorded technical and procedural errors by the theatre team members. We also counted complications and critical incident reports and measured operating time and length of hospital stay. Ethical approval for the study was obtained from the Milton Keynes LREC (ref no 04/Q1603/35). Informed consent was sought from all staff involved in the operations studied, and from the patients undergoing them.

METHODS FOR EVALUATION

Observations were made by researchers (AM and KC) present in theatre throughout each operation. Both had more than 2 years’ experience in theatres as a scientific observer (KC) or trainee surgeon (AM). KC had extensive human factors training and previous experience of methods for observing technical and non-technical skills: he received prior technical and anatomical training about the operations studied, including informal observation of up to 20 procedures before beginning this study. AM received intensive training in scoring non-technical skills from an aviation expert in Crew Resource Management with previous experience of research in operating theatres (TD). Observations were made as ethnographic free-form, real-time notes and written up immediately after the operation.

A single observer was able to observe non-technical skills, technical errors and procedural errors. To assess reliability, both observers recorded scores independently in a proportion of operations. To avoid observer bias in scoring teams after training, an independent third observer (PS), with no previous involvement in the project, co-observed non-technical skills in a sample of post-training cases (n = 11).21

Non-technical skills

Non-technical skills were observed using the Oxford NOTECHS scoring system, developed from a tool designed for aviation personnel,22 via previous studies of non-technical skills and surgical error.19 NOTECHS classifies non-technical skills into four dimensions: (1) leadership and management (L&M), (2) teamwork and cooperation (T&C), (3) problem-solving and decision-making (P&D) and (4) situation awareness (SA). A score between 1 and 4 is assigned for each dimension by semiobjective assessment using prespecified behavioural markers, anchored to four categories (below standard; basic standard; standard; excellent (see box 1)).21 Each of the theatre subteams (surgeons, anaesthetists and nurses) was scored on each dimension. Subteam performance was taken as the sum of the dimension performances (out of 16). Team performance was calculated from the sum of the subteam performance scores (out of 48). Whole team NOTECHS dimension performance was scored as the sum of all subteam performances in that dimension (out of 12). Thus, a score was obtained in each dimension for the theatre team as a whole, and for each subteam. In 36 cases (24 LC and 12 CEA), where two observers carried out independent parallel observations, NOTECHS proved extremely reliable with R_wg = 0.983 overall (see table 1).

Technical errors

Operative technical performance was assessed using the Observation Clinical Human Reliability Assessment (OCHRA) tool, developed for analysis of technical errors during laparoscopic cholecystectomy.2324 Nine stages are recognised, each consisting of a series of subtasks. Errors were recorded as Operative Technical Errors (OTEs). We adapted this method for use in carotid endarterectomy, which was divided into nine mandatory stages and one optional stage.

Procedural errors

We recorded slips, errors and mishaps occurring in theatre but outside the operative field, using a taxonomy developed in earlier studies,19 which we described as Non-Operative Procedural Errors (NOPEs). This category included absence or malfunction of essential equipment, inability to use or set up equipment, dropping items and miscommunications (table 2). Like the Oxford non-technical skills scoring method (NOTECHS), operative technical errors and non-operative procedural errors were also reliably observed (R_wg = 0.987 and 0.831 respectively).

Safety climate assessment

All staff under study were asked to complete the Safety Attitudes Questionnaire25 prior to training (after the initial 6-month observation period) and again after the completion of the coaching period (3 months after the classroom training). The SAQ evaluates knowledge and attitudes relating to workplace safety on six dimensions, of which only one (Teamwork Climate) was directly relevant to the training delivered.

Additional outcome data

The operating time, from patient arrival in theatre to disconnection from anaesthetic support, was recorded for all cases. Postoperatively we recorded: length of hospital stay; any return to theatre or unplanned admission to the intensive treatment unit; critical incident reports; complications recorded in case notes or recalled by the patient during a structured telephone interview conducted a mean of 12 weeks after surgery.

ANALYSIS AND INTERPRETATION

We studied 26 LC operations and 22 CEA procedures before the intervention, and 32 and 23 procedures afterwards. All personnel (n = 83) regularly involved in LC or CEA procedures were invited to the classroom sessions, and 54 (65%) attended. All personnel were involved in subsequent “bedding in” coaching in theatre. After teamwork training, at least two team members had to have attended classroom training to make the case eligible for study.

We compared SAQ scores using paired t tests; non-technical skills scores (Oxford NOTECHS), non-operative procedural errors (NOPEs) and operative duration using unpaired t tests; and operative technical errors and length of stay using Mann–Whitney U tests. Length of stay was also analysed by examining the difference in population percentages with z scores.26 We evaluated agreement between observers using R_wg, and the relationships between technical errors and non-technical skills scores (NOTECHS) using the Spearman correlation. We also compared post-training non-technical skills scores in the presence or absence of a preoperative briefing.

STRATEGY FOR CHANGE

Our classroom training intervention was based on the principles of civil aviation Crew Resource Management (CRM) training. Attendees received 9 h of mixed didactic and interactive teaching. The aims of the training programme were to increase knowledge, change attitudes and improve behaviour in relation to: (a) safety, situation awareness, and error management; (b) self-awareness, communication and assertiveness; and (c) decision-making, briefing and debriefing. The three 3 h modules each concentrated on one of these aspects. In the first module, the theory of organisational error and the value of flat hierarchy, clear impersonal communication and an understanding of situational awareness and “red flags” were explained and demonstrated. In the second module, Myers–Briggs profiles were used to explain personal communication styles, and a tool for graded authority challenge was introduced. In the third module, role-play was used extensively to develop briefing and debriefing skills. A video, role-play scenarios, wall posters and pocket cards containing summaries of the course content were created and distributed (see web material). Following the classroom course, teams were supported in theatre by twice-weekly visits from aviation CRM trainers, who provided encouragement, coaching and feedback during a 3-month “bedding in” period. Typically, trainers observed a briefing, operation and debriefing, and then gave their own feedback to the team and to individuals. They refrained, however, from intervening during these processes, so as to allow local ownership to develop and minimise potential sensitivity among senior staff.

EFFECTS OF CHANGE

The results of measures of attitudes, knowledge, behaviour, technical error and outcomes before intervention are shown in table 3.

Effect of training on safety climate and non-technical performance

Training increased the mean SAQ teamwork climate score (t = −2.95, p = 0.007), while other SAQ components were not significantly affected, in keeping with expectations (see above). Team non-technical skills scores were significantly increased by training (t = −2.35, p = 0.021). LC teams experienced a marked increase in non-technical skills scores after training, but CEA teams did not (table 3). The effects of training were heterogeneous (table 4). Nursing subteam scores increased significantly, whereas anaesthetic and surgical subteam scores did not. Both P&D and T&C increased significantly, whereas SA and L&M did not. The non-technical skills scores of the independent third observer agreed very well with those of the two study observers (R_wg = 0.976).

Effect of training on technical performance

Mean overall technical error rates declined significantly from 1.73 to 0.98 per operation (p = 0.009). The change was more marked for LC, where it achieved significance, than for CEA, where it did not (see above comments). Non-operative procedural errors (NOPEs) were reduced by about 40% after training, which was highly significant for both operations (see table 3). The number of NOPEs per operation was three and 10 times higher than the number of operative technical errors in LC and CEA, respectively.

Interactions between non-technical performance and technical errors

There was a significant but relatively weak association between overall non-technical skills score and operative technical error rate (rho = −0.215, p = 0.024). Surgical subteam non-technical skills score (rho = −0.236, p = 0.013), whole-team SA (rho = −0.300, p = 0.001) and in particular surgical SA (rho = −0.436, p<0.0001) were much more strongly associated with technical error rates than overall team non-technical skills score.

Effect on clinical outcomes

The mean operating time was unaffected by the training (pretraining 103.7 (38.9) min, post-training 101.2 (49.1) min), and the mean LOS was not significantly reduced (pretraining 2.23 (0.7) days, post-training 2.02 (0.6) days).

Five complications were recorded from procedures performed before training, and three from those performed afterwards. One critical incident report was filed in the post-training group. No patient in either group returned to theatre or went to intensive treatment unit after surgery.

NEXT STEPS

Training in non-technical skills resulted in improvement in attitudes to safety, team non-technical performance and technical error rates both in the operative field and outside it. Reductions in hospital stay and complications were not significant, which was unsurprising given the small study numbers. Previous observational studies have shown associations between teamwork, performance and safety in theatre,1319 but this is the first study to show improved technical outcomes after an intervention to improve non-technical skills. The results suggest that team training can improve the performance of theatre teams in a manner which is likely to bring clinical and operational benefits. Our findings are consistent with the MEDTEAMS study of ER staff,17 which reported a significant reduction in clinical errors after team skills training. Further work is, however, required before this type of training can be generally recommended.

The before–after study design we used is at risk from time-related effects on the outcomes of interest, and our study population, while large by comparison with most studies of this type,2728 is small in absolute terms. We could not identify any external factors which changed during the study period and might have produced improvements in performance. We considered this possibility in terms of (a) personnel changes, (b) staffing levels and mix, (c) workload and case-mix, (d) staff morale factors, (e) equipment and resource problems. Factors (c) and (e) did not change during the study. There were few changes in the senior personnel involved. The hospital suffered a financial crisis during the study, which led to a number of staff redundancies, some reorganisation within the theatre teams and a decrease in staff morale. We were not able to train all the appropriate staff members, and a greater effect might have been achieved if training had been universal. Since the observers could not be blinded about which procedures were carried out by CRM trained team, and understood the study hypothesis, there was a risk of observer bias. Observer bias in the Oxford NOTECHS non-technical skills scoring method, the validation of which has been reported elsewhere,21 was controlled for using an independent third observer. The methods used to record errors (operative technical errors and non-operative procedural errors) were semiobjective but highly structured. We cannot exclude a contribution of the “Hawthorne effect” of close observation on performance but would have expected such an effect to be maximal early in the observation period, that is prior to the intervention, since these effects typically fade with time. A summary of the threats to validity in the study and our responses to them is shown in table 5.

Overall, our results support the hypothesis that improving teamwork can improve technical performance, but some details were surprising. We found large decreases in technical error rates associated with apparently modest improvements in non-technical skills, and in the case of CEA a reduction in both technical error and non-operative procedural errors with no change in non-technical skills score. The training programme caused improvements mainly in teamwork and cooperation, problem-solving and decision-making, whereas situation awareness, particularly within the surgical team, was most strongly correlated with operative technical error and did not improve significantly. One interpretation of these findings is that surgical technical performance may have been affected indirectly by the better working environment resulting from improvements in nursing teamwork performance.

We cannot tell whether the NOTECHS non-technical skills scale reflects differences between teams equally well at all points on a scale between worst and best possible performance. Small changes in non-technical skills dimensions might therefore be associated with major improvements in technical performance at one or other end of the Oxford NOTECHS scale. It was interesting to consider why the training was more successful in the LC teams than the CEA team. The CEA team was much smaller and more stable than the LC teams and had a much higher level of baseline non-technical performance. Our subjective impression, supporting the non-technical skills scores, was that the intervention was generally welcomed by nursing staff, but resisted or complied with passively and reluctantly by some senior medical staff, particularly in the CEA team. We postulate that increases in nursing non-technical skills for this team were cancelled out by decreases in the other subteams, but that an improvement in team performance nevertheless occurred, leading to a reduction in technical errors. Whole-team non-technical skills scoring as used in this study may lack sensitivity for those factors most strongly associated with improvements in technical errors, and may therefore require further refinement. Surgical team non-technical skills scores were highly reproducible and generally high, probably because the continuous activity of the team members made inferences about the cognitive aspects of their scores easier. By contrast, it was sometimes difficult to determine whether inactivity or silence in other subteams represented appropriate behaviour, loss of situational awareness or even loss of cooperation. We observed numerous instances which illustrate the need for improved communication, and the difficulty of accepting an open learning approach in the current surgical culture (see boxes 2–4). The power of hierarchy in inhibiting free communication was made particularly evident by observed examples of this type.

Our results are necessarily specific for one training programme, and further work will be needed to determine the nature and content of the most effective training intervention for reducing errors in theatre. We were impressed by the value of the “bedding-in” period, which allowed willing teams to develop competence and confidence by live practice with expert mentorship. Initially, the simple presence of the mentors was often sufficient to stimulate post-training teams to begin with a briefing, for example, whereas teams with no mentor present were likely to forget. We recommend that future training programmes incorporate prolonged on-the-job support. The aspects of the training which produced most subjective evidence of discomfort and difficulty were open honest debriefing and the use of a graduated challenge method to transmit urgent situational awareness information to authority figures. These appear to run counter to the prevailing professional culture, and are unlikely to be sustained without continuing direct efforts to promote them, as well as a general safety strategy designed to promote culture change. We suspect this will require strong, well-publicised institutional support and action at multiple levels, including education of new trainees, development of “champions” and substantial commitment of institutional resources to the development of official policies, incentives and regular retraining.29^–31 Anecdotal evidence from aviation experience suggests that management action to ensure compliance via policies and procedures may be necessary to ensure sustainable adoption in the face of cultural resistance. Such comprehensive action could yield further improvements in safety in the context of a formal quality-mprovement programme. A larger trial with a randomised design and parallel explanatory qualitative analyses is needed to determine whether our findings can be reproduced in different settings, and whether clinical benefit can be directly demonstrated.