Article Text
PDF
Abstract
Objective Paediatric cardiac surgery has a low error tolerance and demands high levels of cognitive and technical performance. Growing evidence suggests that further improvements in patient outcomes depend on system factors, in particular, effective team skills. The hypotheses that small intraoperative non-routine events (NREs) can escalate to more serious situations and that effective teamwork can prevent the development of serious situations were examined to develop a method to assess these skills and to provide evidence for improvements in training and performance.
Methods This mixed-method design, using both quantitative and qualitative measures, relied on trained human factor observers who observed and coded NREs and teamwork elements from the time of patient arrival into the operating room to patient handover to the intensive care unit. Real-time teamwork observations were coupled with microsystem preparedness measures, operative duration, assessed difficulty of the operation and patient outcome measures. Behaviour was rated based on whether it hindered or enhanced teamwork.
Results 40 paediatric cardiac surgery cases were observed. Surgeons displayed better teamwork during complicated procedures, particularly during the surgical bypass/repair epoch. More procedural NREs were associated with a more complicated postoperative course (Muncomplicated=9.08; Mminor complications=11.11; Mmajor morbidity=14.60, F(2,26)=3.46, p<0.05). Procedural NREs decreased substantially over time (M1=13.5; M2=7.1, F(1,37)=33.07, p<0.001).
Conclusions Structured observation of effective teamwork in the operating room can identify substantive deficiencies in the system and conduct of procedures, even in otherwise successful operations. High performing teams are more resilient displaying effective teamwork when operations become more difficult.
- Patient safety
- cardiac surgical procedures
- microsystem
- teamwork
- non-routine events
- non-technical skills
- adverse event
- human factors
- patient outcomes
Statistics from Altmetric.com
- Patient safety
- cardiac surgical procedures
- microsystem
- teamwork
- non-routine events
- non-technical skills
- adverse event
- human factors
- patient outcomes
In recent years, the relationship between surgical team behaviour and patient outcome has been studied by a number of researchers.1–9 Research on teams generally distinguishes between taskwork and teamwork.10 Taskwork concerns the individual knowledge, skills and attitudes, whereas teamwork concerns the communication and coordination processes that are required to bring together the individual knowledge, skills and attitudes in the service of a common and valued team goal. Individual surgical team members are highly specialised and have their own functional taskwork (eg, anaesthesia, nursing, surgery and perfusion), yet come together as a team towards the common goal of treating the patient. This ‘working together’ of a clinical microsystem11 is accomplished by a complex suite of ‘non-technical skills’.12–16
Teams that score low on independently observed non-technical skills make more technical errors17 and in cases where teams infrequently display team behaviours, patients are more likely to experience death or major complications.9 Davenport et al18 found a significant correlation between subjective assessment of teamwork by team members themselves and postoperative morbidity. Good teamwork (in terms of both quality and quantity) is associated with shorter duration of operations, fewer adverse events and lower postoperative morbidity.3 6 8 9 18
However, these studies suffer from a number of drawbacks. First, all studies are observational in nature, drawing conclusions about associations between teamwork, adverse events and patient outcomes. From these study methodologies, it is impossible to determine whether effective teamwork causally determines adverse events and patient outcomes. Second, in most studies, the observations of teamwork were made immediately at the end of the operation or at sparse intervals during the operation. In the first case, knowledge of the outcome may have influenced the coding of teamwork, so-called ‘hindsight bias’.19 In the second case, the frequency of observations of teamwork was too low to permit robust conclusions. Third, a conceptual limitation is that patient outcomes are not solely determined by the teamwork of the surgical team but also by patient-related factors, postoperative intensive care treatment or ‘chance’. Brown et al20 and Lilford et al21 recommend using one or more surrogate end points for patient outcomes.
The goal of the current study is to assess the relationships between the complexity of the surgical operation, duration of the procedure, level of preparedness of the team, non-routine events (NREs), teamwork and the postoperative outcomes.
Methods
This prospective, observational study of intraoperative events in a paediatric cardiac surgery (PCS) microsystem was conducted at University Medical Center Utrecht.22 The study involved structured observations of the PCS microsystem personnel (ie, surgeons, anaesthetists, nurses and perfusionists) during two periods, from December 2008 to March 2009, and from September 2009 to November 2009. We studied a particular team in detail over a period of 11 months, observing 40 different operations using the team as its own control point. We observed the team in continuous measurements throughout the full perioperative journey in order to minimise the risk of hindsight bias. The study received ethics committee approval and consent by all team personnel. The team consisted of eight to nine personnel and was relatively stable. Although the exact team composition could vary from operation to operation, individual members performed the same roles in different operations.
The clinical case complexity was measured using the basic Aristotle risk assessment scoring system.23 This scoring tool stratifies risk based on the potential for morbidity, mortality and the anticipated technical difficulty of a given procedure.
MiniSTAR (Surgical Team Assessment Record): preoperation and postoperation questionnaires
We developed and piloted a very brief four-item questionnaire adapted from De Leval et al8 completed by each member of the PCS team prior to, and after, each operation. The questionnaires were de-identified except for the provider type. The preoperation miniSTAR consisted of the following questions:
Did you receive, in your opinion, sufficient information about the child scheduled for the operation?
Did you sleep well last night?
Are you troubled or distracted by any physical or mental stress that could possibly affect your performance during the operation?
Are you concerned about the performance of other team members during this operation?
The scores of the four questions were collapsed into one score, with a minimum of 0 (indicating optimal preparation) and a maximum of 7 (indicating serious concerns about team preparedness and readiness).
The postoperation miniSTAR was completed immediately after the operation. It consisted of the following questions:
Did you do anything that should not have been done or done differently, even though this may not at all have led to patient harm?
Did you notice that other team members did something that had better not been done or done differently, even though this may not at all have led to patient harm?
Did you notice that you or others had done anything that should not have been done or done differently, and which has occurred before?
Was the operation carried out in a pleasant and harmonious atmosphere?
Did you notice any conflicts among team members and how were they resolved?
These five questions were collapsed and coded into one score, with a minimum of 0 (indicating all went well) and a maximum of 6 (indicating major NREs and disturbances). The pre- and post-miniSTAR scores were averaged within each subteam, and subsequently combined across subteams in order to arrive at a total score for the team as a whole for each operation; (note: due to fewer response options on the post-miniSTAR, the maximum score obtainable was lower than the maximum score on the pre-miniSTAR).
Observations of teamwork classification tool and rating scale
Our observational methods and tools were described extensively in our companion paper,24 thus only a brief summary is provided here. Communication among team members was noted and documented on a scoring form in real time. After each operation, we classified teamwork aspects using four main categories: leadership, situation awareness, decision making, and teamwork and cooperation, based on a validated rating instrument.12–14 Each teamwork aspect was also rated on a 7-point Likert scale in terms of whether it hindered or enhanced teamwork.25 This yielded a ‘teamwork quality’ score. Teamwork was classified and rated separately for each discipline (ie, surgeons, anaesthetists, perfusionists and nurses). For each operation, a total teamwork quality score based on the averages of all four subteams (that were in turn based on the averages of all four main categories) was calculated.
Documentation of NREs
NREs were defined, in a definition adapted from the nuclear power industry as ‘any event that is perceived by care providers or skilled observers to be unusual, out-of-the-ordinary or atypical.’26 This is a broad definition and includes everything from phone calls, masks not worn properly to serious incidents endangering the patient's well being. Hence, our definition was expanded to include elements of a normative task analysis we conducted using process mapping specifying the norms and tasks of each operation.27 28 Given the relative frequency of events such as, equipment problems and mask violations, these events cannot be regarded as ‘unusual’. The NREs observed were corroborated by team members by asking about NREs in the post-miniSTAR questionnaire, and by interviewing team members in order to clarify the nature of the NREs.
All NREs were categorised later by two senior anaesthetists on the research team into one of the following four categories:
Disturbances during the execution of the procedure (eg, wrong size cannula; no medication labels on infusion lines, empty oxygen cylinder)
Remarkable individual behaviour, unrelated to the procedure (eg, eating apple, drinking coffee, not wearing mask properly, arriving late due to traffic jam)
External events (eg, pagers and beepers, phones ringing; no ICU beds available)
Events caused by unpredictable patient factors (eg, allergic reactions, inflammation of eye, pulmonary hypertension)
The training and inter-rater reliability was reported in our companion paper. The inter-rater reliability assessed at three intervals throughout the study showed a Cohen κ score of 0.77, 0.50 and 0.66 (26).
Outcome measures
The postoperative course was determined on the basis of 30-day postsurgical outcomes and classified into three categories: uncomplicated, minor complications, major complications or death.
Results
A comparison of study measures
We first present the descriptive statistics of the variables measured (see table 1).
A comparison of main study measures
The patient outcomes were measured as 25 uncomplicated cases, 9 minor complications, 5 major morbidity cases and data missing from one operation.
Relationship between teamwork and NREs
Previous research has demonstrated that good teamwork is associated with fewer NREs, we were therefore interested in the relationship between the quality of teamwork, as assessed on the non-technical skills rating scale, and the number of observed NREs. No correlation could be found between the leadership, situation awareness, teamwork and cooperation domains for all of the subteams, and the number of NREs. However, there was a significant correlation between surgical decision making during cardiopulmonary bypass and the number of NREs, r=0.66, p<0.01. This finding suggests that, as the number of NREs during this phase increased, the surgeons scored higher on decision making. It should be noted that this result is based on an N of 18.
Relationship between patient outcomes and NREs
We grouped NREs into four categories: 1. procedural; 2. individual; 3. external; 4. patient. Using an ANOVA with three outcome categories, we found a significant difference in occurrence across the patient outcome categories for the procedural category: Muncomplicated=9.08; Mminor complications=11.11; Mmajor morbidity=14.60, F(2,26)=3.46, p<0.05. None of the other categories of NREs reached significance.
Relationships between teamwork and complexity, post-miniSTAR and patient outcomes
Using an ANOVA with three outcome categories and four teamwork categories, the quality of teamwork and cooperation by surgeons increased as patient outcomes worsened: Muncomplicated=3.19; Mminor complications=3.44; Mmajor morbidity=3.28, F(2,36)=3.85, p<0.05 (η2=0.18). The quality of teamwork for the other subteams was not significantly different for the different outcome categories.
The correlation between teamwork scores and the post-miniSTAR was −0.32, p<0.05. A further inspection of this correlation revealed that the perfusion subteam was responsible for this effect: their team performance correlated significantly with the post-miniSTAR, r=−0.42, p<0.01, whereas the other subteams showed no significant correlations. Therefore, the better the teamwork of the perfusion subteam, fewer complications were encountered during the operation.
The post-miniSTAR score averaged across all subteams differed significantly for the three outcome categories: Muncomplicated=2.18; Mminor complications=4.05; Mmajor morbidity=5.47, F(2,26)=7.03, p=0.004. A further breakdown into different subteams showed that the different outcome measures were associated with different post-miniSTAR scores for the anaesthetists, F(2,29)=3.91, p=0.03, and the perfusionists, F(2,28)=9.89, p<0.001, but not for the surgeons and nurses (both p values >0.05).
Longitudinal changes
Analyses were carried out by comparing the first set of observations with the second set of observations. With the exception of the number of NREs, no significant differences were found in any of the measures, including the Aristotle score, which suggests an identical case mix in both clinical sets. The mean number of NREs per case dropped from 20.3 to 11.1, F(1,37)=33.93, p<0.001. This drop was particularly significant for the NREs related to the execution of the procedure (M1=13.5; M2=7.1), F(1,37)=33.07, p<0.001. As shown in figure 1, with the exception of the first two observations of the second set, all operations showed fewer than 15 NREs. By contrast, all operations in the first set, with the exception of four operations, encountered more than 15 NREs.
Number of non-routine events that occurred during each operation.
Discussion
We found that complex operations take longer and lead to more NREs, particularly during the surgical bypass and repair phase. Good surgeons seem to display more effective decision-making skills, particularly during these non-routine situations. Yet, increased complexity is associated with poorer patient outcomes and higher subjectively experienced adversity. We found that a brief questionnaire completed immediately after the operation was predictive of patient outcomes, particularly when filled out by the anaesthetists and perfusionists. In order to avoid interpretation problems with the correlation between surgical performance during the operation and extended stay in the ICU, this questionnaire may be taken as a proxy measure for patient outcomes after the operation.
The puzzling result in this study, as compared with previous studies, is that good teamwork by surgeons is associated with poorer patient outcomes rather than lower postoperative morbidity. We believe this might reflect a flaw in the methods of previous studies that rated teamwork mostly in hindsight, with knowledge of the outcome in mind. We hypothesise that during difficult operations teams will differ, that is, poor teams will display ineffective teamwork, whereas excellent teams (such as the one we observed) will display good teamwork. If a surgeon communicates more explicitly during a difficult operation, his or her teamwork may be rated higher, but the operation itself is still likely to result in adverse outcomes. Given this line of reasoning, we would expect better teamwork scores to be associated with poorer patient outcomes. In our opinion, teamwork processes are adaptive mechanisms that emerge primarily when operations become more difficult, and are less noted during less complex operations.
Limitations of this study
This study was limited insofar as it studied one PCS microsystem. Objective performance results (ie, European Thoracic Society outcomes database) have shown that this particular team has higher than average patient survivability rates while completing, on average, more complex operations than other European paediatric cardiac surgical teams.
A novel contribution of our research has been the observation of a single team over an extended period of time. Several changes in practice occurred during the study period that we could not control for. For instance, after our first set of 19 operations, a briefing procedure was introduced (modelled after the WHO Surgical Safety Checklist).29 We conducted a meeting with the team in which we discussed the outcomes of the first set of operations. A few months later, the surgical unit was confronted with a large increase in case load (shifting from an average of three cases each week to an average of five cases each week), due to the closing down of another surgical unit and the uptake of patients from this unit. As it happens, both events occurred during a period where we were not actively observing.
An intriguing finding was the large decrease in NREs, particularly those concerned with the execution of the procedure. This decrease was not due to differences in complexity (case mix), duration or patient outcome, as these measures did not differ from the first set of observations to the second. One possibility is that this effect was caused by different rating behaviours on the part of our observers. However, as shown in figure 1, observers showed identical observation patterns during both sets of observations, making observer bias less likely. However, another possibility is that the decrease was influenced by the increased attention of the team itself to the occurrence of NREs. A third possibility is that the introduction of the briefing procedure in between the two observation periods has led to increased awareness to NREs and thus improved patient outcomes as was reported in another study.30 Given that our study was carried out without a control group, it is impossible to draw firm conclusions in this respect.
References
Footnotes
Linked article 040105.
Funding TNO Quality of Life, Wassenaarseweg 56, Leiden, The Netherlands.
Competing interests TS and FH are employed by the University Medical Center Utrecht and were part of the team under study.
Ethics approval Full institutional ethics approval was obtained and consent was acquired from all PCS team members. Ethics approval was provided by IRB, University Medical Centre Utrecht.
Provenance and peer review Not commissioned; externally peer reviewed.