Article Text
PDF
Abstract
Background Improving patient safety by training teams to successfully manage emergencies is a major concern in healthcare. Most current trainings use simulation of emergency situations to practice and reflect on relevant clinical and behavioural skills. We developed TeamGAINS, a hybrid, structured debriefing tool for simulation-based team trainings in healthcare that integrates three different debriefing approaches: guided team self-correction, advocacy-inquiry and systemic-constructivist techniques.
Methods TeamGAINS was administered during simulation-based trainings for clinical and behavioural skills for anaesthesia staff. One of the four daily scenarios involved all trainees, whereas the remaining three scenarios each involved only two trainees with the others observing them. Training instructors were senior anaesthesiologists and psychologists. To determine debriefing quality, we used a post-test-only (debriefing quality) and a pre-post-test (psychological safety, leader inclusiveness), no-control-group design. After each debriefing all trainees completed a self-report debriefing quality scale which we developed based on the Debriefing Assessment for Simulation in Healthcare and the Observational Structured Assessment of Debriefing. Perceived psychological safety and leader inclusiveness were measured before trainees’ first (premeasure) and after their last debriefing (postmeasure) at which time trainees’ reactions to the overall training were measured as well.
Results Four senior anaesthetists, 29 residents and 28 nurses participated in a total of 40 debriefings resulting in 235 evaluations. Utility of debriefings was evaluated as highly positive. Pre-post comparisons revealed that psychological safety and leader inclusiveness significantly increased after the debriefings.
Conclusions The results indicate that TeamGAINS could provide a useful debriefing tool for training anaesthesia staff on all levels of work experience. By combining state-of-the-art debriefing methods and integrating systemic-constructivist techniques, TeamGAINS has the potential to allow for a surfacing, reflecting on and changing of the dynamics of team interactions. Further research is necessary to systematically compare the effects of TeamGAINS’ components on the debriefing itself and on trainees’ changes in attitudes and behaviours.
- Anaesthesia
- Communication
- Crew resource management
- Simulation
- Team training
Statistics from Altmetric.com
Background
Improving patient safety by training teams to successfully manage emergencies is a major concern in healthcare.1–9 Most current trainings use simulation of emergency situations to practice and reflect on relevant clinical and behavioural skills.10–18 A core element of these simulation-based trainings is the debriefing, that is the after-action review of the simulation scenario.19–21 This debriefing is an instructor-guided conversation among trainees that aims to understand the relationships among events, actions, thought and feeling processes, and performance outcomes of the simulation.19 ,21–23 That is, these group-based conversations aim at promoting experiential learning and developing strategies that can be applied in future performance episodes.16 ,20 ,24 ,25 Yet, evidence as to how such debriefings should be guided by the instructor so that trainees benefit most effectively is just beginning to emerge.19 ,23 ,26–28 The purpose of this paper is to describe the development of an integrated debriefing approach and demonstrate how trainees perceive this approach. We will present this approach after a short review of the debriefing literature. This will be followed by an empirical study in which we evaluated the integrated debriefing approach during simulation-based trainings of clinical and behavioural skills for anaesthesia teams.
Debriefings in simulation-based trainings
The potential of debriefings for success or failure of simulation-based trainings is not yet fully explored.29 ,30 Three important trends in the literature on debriefings suggest avenues for enhancements to existing debriefing techniques.
First, the degree to which debriefings should be instructor-guided or learner-guided is an area of debate. For example, there is growing recognition that trainees are aware of behavioural processes such as teamwork and communication and benefit more from being supported in articulating, exploring and learning from these processes rather than from being merely taught how well they worked as a team or how poorly they communicated.29 ,31 However, the role of the instructor in surfacing and changing these processes and mental models is crucial.19
Second, there is growing recognition that conducting good debriefings requires skills to elicit and support trainees’ understanding of the processes that took place in the simulated scenario—skills in which instructors should be trained.16 ,29 ,31 ,32
Third, best practices for conducting debriefings have been suggested.22 ,28 ,29 ,31–33 They include the creation of a supportive learning environment, the training of instructors and the focus on few but critical performance issues during the debriefing process.22 ,34 More specifically, debriefing techniques have been developed that provide instructors with structure and techniques for effective debriefings. We consider three of them as particularly promising: (1) guided team self-correction,35 (2) advocacy-inquiry19 and (3) systemic-constructivist techniques.36 We will briefly describe them in the following and explain why we regard them as useful in simulations-based trainings in healthcare.
The first approach, guided team self-correction, provides structure and specific techniques aiming at self-correction of the team.35 The debriefing is organised around a prespecified model containing relevant teamwork skills. Following this model, an instructor asks the team to describe positive and negative instances of their performance during simulation that illustrate the components of the model (eg, ‘Give me an example of when priorities were clearly and appropriately stated.’).35 Thereby, a critical and systematic self-analysis of the trainees is fostered while the instructor takes a non-judgmental and neutral position. Still, the instructor can share his/her own opinion and observations but s/he is taught to wait until the team members have offered their input (Smith-Jentsch, personal communication, 2012).
The second approach, advocacy-inquiry, combines feedback and reflective practice.19 ,37 ,38 It is more instructor-led than guided team self-correction and offers an approach for expert judgment by deliberately voicing performance gaps and questioning trainees’ respective taken-for-granted assumptions and mental routines.19 The approach asks instructors and learners to be transparent about their thought processes. For example, an instructor would say ‘I saw you re-attempting to intubate using the laryngoscope three times in a row, each time it turned out unsuccessful. I think that you could have intubated faster by using another device such as the Laryngeal Mask or Bag Mask Ventilation. So I am wondering what was on your mind?’
The third approach, systemic-constructivist debriefing, is based on family systems theory and constructivism as they are used in systemic therapy.36 ,39 ,40 It has only very recently been introduced as a promising debriefing approach and has received little attention so far.36 In its core, systemic therapy focuses on individuals within their systems, that is, it looks at patterns and dynamics of interactions and relationships rather than on isolated individual behaviour.41 Therefore, using techniques of systemic therapy seems promising for exploring the relationships among events, actions and performance outcomes of the simulation—which is the main purpose of the debriefing. Prominent examples of systemic techniques are circular questions and the Reflecting Team.
Circular questions aim at exploring a dyadic relationship as it is seen by a third person by inviting the third person to describe the relationship of two others in their presence.42 ,43 For example, the instructor may ask the nurse: ‘What did the senior physician do when she entered the operating room (OR) and how did the resident physician react to that?’ Thus, circular questions ask trainees to circle back and comment from an outside perspective on an interaction in which they took part. It allows people within a team to track team behaviour patterns, generate new information and foster perspective taking in the debriefing. Circular questions mostly look at specific interactions in terms of differences in behaviour and not in terms of characteristics supposedly intrinsic to the individual.42 Thereby, they provide a suitable means to discuss how the context had ‘made someone do something’ and to reflect on the correspondence bias, that is ‘the tendency to draw inferences about a person's unique and enduring dispositions from behaviours that can be entirely explained by the situation in which they occur’. (ref. 44, p. 21)
The Reflecting Team,40 ,45 as used in family therapy, usually consists of members of the clinical team who would observe the clinical interview through a one-way screen. During an interview mid-session break, they would comment on the interview process in a positive, respectful and non-accusatory language (eg, highlighting strengths of the family, offering different explanations for behaviours, suggesting new possibilities for problem solution) while the family and the interviewing therapists listen to them.40 That is, the Reflecting Team is collaborative in nature, builds on pre-existing strengths and offers not-yet-thought-of understandings of what has been defined as problematic and how it can be dealt with.40 ,46 It has already been successfully used for teaching clinical skills to family practice residents.47 In many simulation-based team trainings only some of the trainees participate in one scenario at a time while the remaining trainees and the instructors observe them from another room via synchronised video broadcast. Thus, the remaining two groups would constitute a Reflecting Team during the debriefing and discuss how the events, actions and performance outcomes of the simulation could have been interrelated and even generate new possible solutions for crisis management.36 Thus, in the debriefing they could provide a useful resource of information for those trainees who had participated in the simulation and could therefore be systematically involved in the reflection process.
As these three debriefing approaches—guided team self-correction, advocacy-inquiry and systemic-constructivist debriefing—each have genuine and particular advantages (see online supplementary table 1), we think that combining them into a hybrid debriefing tool, namely TeamGAINS (Guided team self-correction, Advocacy-Inquiry, Systemic-constructivist), would contribute to realising the full potential of the debriefing. TeamGAINS would allow for using elements of each of the approaches in an integrative and adapted fashion.
For example, from our perspective guided team self-correction may be particularly useful when a generic, prescriptive expert model of teamwork is the learning objective, whereas advocacy-inquiry may be especially useful for sharing one's point of view and for exploring individual frames, while systemic-constructivist techniques may be useful for surfacing and changing the dynamics of team interactions. For instance, when one team member starts complaining about his or her colleague, using a solution-oriented, circular question of systemic debriefing could substitute an open inquiry and redirect the conversation in a constructive mode that explicitly addresses the interaction between the team members (eg, ‘What would she have needed from him in that situation?’).48 Likewise, if the learning objective was ‘preventing fixation errors’,49 observer-perspective circular questions50 could be used for surfacing how fixation can be noticed and changed, for example, ‘I would like to talk about fixation errors. Let's assume, hypothetically, your colleague would get fixated. I am wondering how you would recognise it? (…) What are your thoughts on what your colleague may need from you to get out of this fixation?’ Precisely how the three approaches should be used together cannot and should not be standardised because it is a function of the instructors’ experience, the trainee's experience, and the learning objectives of the simulated case.29 However, we propose that they complement each other and suggest situations in which the use of one of the three approaches may be particularly indicated based on its special advantages (see online supplementary table 1, supplementary figure 1).
In the following, we will introduce a debriefing structure that combines the advantages of guided team self-correction, advocacy-inquiry and the systemic-constructivist approach. Based on previous debriefing work,19 ,21 ,22 ,27 ,35–37 51–57 we have developed this debriefing structure for trainings aiming at teaching clinical as well as behavioural skills as we think they should best be trained integratively.20
TeamGAINS: the integrated approach for structured debriefings
Debriefing content
The contents of the debriefing are the clinical and behavioural skills required in the simulated task. For example, if an unexpected difficult intubation is simulated, the clinical skills rely on guidelines for managing unanticipated difficult airways58 and the behavioural skills rely on those cognitive and social skills that are particularly required in that situation (eg, call for help early, prevent and recognise fixation errors, re-evaluate the situation). Task analysis is used for determining what clinical and behavioural skills are required throughout the scenario.59 A list of 10 general behavioural skills which we adopted from Crisis Resource Management (CRM) principles suggested by Gaba et al,14 ,60 Fletcher et al,7 and empirical research on teamwork in anaesthesia61 ,62 is introduced in the simulation pre-briefing and displayed visibly in the debriefing room (table 1). That is, the clinical and behavioural skills serve as an expert model around which the debriefing is organised.63
Trained Crisis Resource Management principles
Debriefing procedure
To discuss clinical and behavioural skills, each debriefing is facilitated by a senior anaesthesiologist and a psychologist. Using the videotaped performance during the simulated scenario, they guide the debriefing by following the six steps described next. With these steps we follow excellent previous recommendations on the optimal flow of debriefings.21 ,22 ,27 ,36 ,37 ,51–57 All steps, including examples of instructors’ communication and sources in the debriefing literature, are also shown in table 2. Online supplementary figure 1 provides a guide as to how to combine guided team self-correction, advocacy-inquiry and systemic-constructivist techniques.
Guided team self-correction, Advocacy-Inquiry, Systemic-constructivist: the integrated approach for structured debriefings
In Step 1, the instructors announce that each debriefing begins with an initial round in which everyone can describe their feelings and reactions during the simulation.27 ,36 ,37 ,51 ,55 ,56 The instructors then start off asking each scenario participant ‘How did you feel?’ This open-ended, narrative question usually invites a brief statement about perceptions, emotions and cognitions the trainees experienced during the simulation. One goal of this step is to allow each trainee to release the tension built up during the scenario and to enable them to concentrate on an unemotional reflection of their actions during the further debriefing. Another goal of this step is to obtain information about what mattered to the trainees during the case.37 The instructors merely acknowledge but do not comment on the trainees’ answers. If the trainees’ comments refer to specific clinical and behavioural skills, the instructors would explain that these were important aspects that would be taken up in the following discussion. By the end of Step 1, each trainee should feel ‘ready for reflection’.
In Step 2, the clinical part of the scenario is discussed.27 ,36 ,52 ,56 The goal of this step is to establish a thorough understanding of the clinical procedures and medical problems involved in the scenario. Usually, the anaesthesiologist-instructor starts off with asking: ‘what happened?’, which is followed by the trainees discussing the simulated medical problem. The anaesthesiologist-instructor then provides more detailed case information and, if possible, for the purpose of normalisation64 and demonstration of his/her willingness to take an interpersonal risk19 ,65 shares a brief personal story with respect to the case and his/her reactions to it. Guided team self-correction, advocacy-inquiry and systemic-constructivist techniques are then used as shown in table 2 and online supplementary table 1. By the end of Step 2, the medical problem involved in the scenario and the required clinical procedures should be sufficiently understood. As a result, false or insufficient medical knowledge is detected and revised.
In Step 3, the instructors establish a transfer from simulation to reality.27 ,36 ,52 ,66 The goal of this step is to explicitly link the phenomena of the simulated scenario to the real clinical work.
In Step 4, the instructors reintroduce the expert model and initiate a systematic discussion about the behavioural skills and their relationship to clinical performance.37 ,51 ,52 ,55 ,56 The goal of this step is to establish a thorough understanding of how the behavioural skills, team processes, clinical procedures and performance outcomes of the scenario were interlinked. Again, guided team self-correction, advocacy-inquiry and systemic-constructivist techniques are used as shown in table 2. If available, the instructors also initiate linking the aspects mentioned by the trainees in Step 1 to the respective behavioural skills.
In Step 5, the instructors initiate a summary of the debriefing.27 ,37 ,51 ,55 ,56 The goal of this step is to allow the trainees to develop a clear learning experience from the scenario.
In Step 6, we submit that clinical skills could be practiced again if required, that is, either if the instructors have perceived a performance gap or if the trainees explicitly wish to practise it.67 ,68 For example, the instructor demonstrates the intubation via the Intubating Laryngeal Mask or the correct use of the defibrillator and the trainees subsequently practise this procedure under supervision.
We applied and evaluated the integrated debriefing approach during a simulation-based training for clinical and behavioural skills in anaesthesia. Ideally, an evaluation design would include measures of a potential increase and transfer of skills and of an impact on actual results (eg, team performance).69 However, it has also been acknowledged that conducting evaluations in the field is difficult and simpler evaluation designs have been suggested.70 ,71 In this study, we used a post-test-only no control group design to gain first insights into the usefulness of TeamGAINS by applying a self-report debriefing quality scale. To test the convergent validity of this scale we assessed trainees’ overall reaction to the training. In addition, we used a pre-post no control group design to explore how using TeamGAINS during a simulation-based course would relate to psychological safety and leader inclusiveness—two concepts which are critical for team performance, organisational learning and patient safety.65 ,72 ,73 Psychological safety is the shared belief held by team members that they are safe for interpersonal risk taking.65 As TeamGAINS should provide trainees with the opportunity to openly discuss task work and teamwork, we assumed that perceived psychological safety would increase during the training day. Leader inclusiveness describes the degree to which team members feel that the team leader includes them in relevant decisions and actions.72 As particularly the systemic-constructivist techniques of TeamGAINS aim at surfacing and changing behaviour as a dynamic, interactive phenomenon of mutual influence among individuals (see table 2 and online supplementary table 1) and at adaptive leadership within the team, we assumed that perceived leader inclusiveness would increase during the training day as well. Measuring how psychological safety and leader inclusiveness as two relevant team-related attitudes change via trainings is an important aspect of training evaluation.7 ,69 ,70
Methods
The study was approved by the Ethics Committee of the Canton Zurich, Switzerland. Written consent was obtained from all study trainees.
Setting
The debriefing was administered during a full-day simulation-based training course. The course was designed as combined clinical and behavioural skills training for anaesthesia staff (doctors and nurses). It took place in the newly built simulation centre of a Swiss teaching hospital equipped with the SimMan, a full-scale patient simulator (Laerdal, Starvanger, Norway). Training instructors were two senior anaesthesiologists, one experienced anaesthesia resident whose promotion to senior anaesthesiologist was imminent and two psychologists specialised in human factors and debriefing techniques. All debriefers were intermediate simulation faculty and had, together, more than 500 h of debriefing experience. Together they had conducted debriefings in the development phase of TeamGAINS and were familiar with its structure. The first author, who has a background in systemic family therapy and is very familiar with the systemic-constructivist technique, was present during all debriefings and monitored the correct use of TeamGAINS. Each debriefing was led in a co-debriefing setting by one senior anaesthesiologist and one psychologist with the experienced anaesthesia resident as additional support when available.
Each day, three resident physicians and three anaesthesia nurses participated in the training. On three training days, a senior anaesthesiologist joined them in the training. Figure 1 provides an overview of the training procedure and trainees. The entire study period involved 10 training days. The training day consisted of an introduction into CRM and into clinical and behavioural skills, of a familiarisation with the SimMan, and of four simulated scenarios and subsequent debriefings. All simulated scenarios involved unexpected critical events such as difficult intubation, cardiac arrest or tension pneumothorax in order to simulate crisis management and to increase clinical and behavioural skills requirement. Each day, the first scenario consisted of a very complex scenario (ie, pregnant woman with a polytrauma) which involved the participation of all trainees and served as ‘de-freezer’ for reducing trainees’ fears of being observed in the simulator. The subsequent three scenarios each involved only two trainees (ie, one resident, one nurse) while the remaining trainees observed them from the debriefing room via synchronised video broadcast. That is, three two-person teams each participated in a different scenario that lasted approximately 20 min. Figure 2 provides an overview of the simulation training setting. Video and vital parameter recordings were obtained using the Laerdal setup allowing for synchronised recording and playback of video, monitor and ventilator data. The debriefings took place immediately after the simulation and followed the procedure described in table 2. They lasted about 45 min and involved all trainees, that is, those who actively participated in the scenario and those who observed them.
Overview of the training procedure for each day following a detailed introduction and familiarisation with the simulation environment.
Overview of the simulation training setting.
Study participants
The trainees were 4 senior anaesthetists (2 female and 2 male), 29 resident anaesthetists (11 female and 18 male), and 28 anaesthesia nurses (22 female and 6 male) from a teaching hospital in Switzerland. Training participation was offered by senior management and scheduled based on availability of anaesthesia staff. All participants attended the training voluntarily, during regular working hours and received credits for their participation. Each day, three residents and three nurses participated; on three training days four senior anaesthesiologists additionally took part. This resulted in 61 trainees over the 10-day study period. Organised in teams of two (one resident, one nurse) they participated in the scenario while being observed by their colleagues, except for the first scenario of the day in which all team members participated at once. That is, each trainee actively participated in two scenarios and observed two more.
After each of the daily four debriefings, all trainees were asked to evaluate the debriefings using the measures described below. This resulted in four measurement points of debriefing quality for 52 trainees and three measurement points for nine participants, who had to leave early but were nevertheless included in the analyses, resulting in 235 assessments of the debriefings.
Measures
In order to assess the quality of the debriefings we measured the trainees’ perception of the quality of each debriefing by using a self-report debriefing quality scale.
Self-report debriefing quality scale
We developed the self-report debriefing quality scale based on two valid debriefing evaluation measures: The Debriefing Assessment for Simulation in Healthcare (DASH)26 and the Observational Structured Assessment of Debriefing (OSAD).33 We based our debriefing quality on these measures because they are in line with current training-related learning models,70 quantify the quality of a debriefing in a comprehensive way and explicitly focus on the utility and applicability of the debriefing rather than merely assessing whether trainees liked the debriefing—this is an important advantage because utility judgements are more strongly related to learning and performance than affective reactions.74 We decided not to use the original measures because (A) both had existed in English only and because (B) we assumed that as behaviourally anchored rating scales they would have required more time to complete than we had available between the four simulated cases. Instead, we decided to merge the components of DASH and OSAD into a self-report debriefing scale to capture learners’ reactions to the debriefing as comprehensively as possible within the time constraints given in our simulation training environment. Our intention was to assess as many facets of this reaction as possible. Therefore, it seemed plausible not to solely use a single measure but two measures which, although significantly overlapping, have the potential to grasp a broader spectrum of debriefing quality. We used the wording of the six elements of the DASH Student Version26 (eg, ‘The instructor set the stage for an engaging learning experience.’) and the wording of the maximum ratings of the eight facets of the OSAD33 (eg, ‘Establishes and maintains rapport throughout; uses a non- threatening but honest approach, creating a psychologically safe environment’) as basis for phrasing the German items. All items including their respective sources are depicted in table 3. Each item was rated on a 5-point Likert scale anchored at 1 (strongly disagree) and 5 (strongly agree). With respect to reliability, we tested this scale for internal consistency which was fairly high across all four measurement points (all Cronbach's α>0.90). With respect to validity, we assumed that the scale would have sufficient content validity because we designed it based on two validated measures. We also tested for convergent and discriminant validity. Convergent validity was tested by investigating the scale's correlation with the overall reactions to the training. We assumed that the assessment of the debriefing quality would be closely related to the overall reaction to the training and expected a positive correlation between these measures. Discriminant validity was tested by investigating the scale's correlation with the measures of psychological safety and leader inclusiveness. We assumed that the assessment of the debriefing quality would be neither closely related to team psychological safety nor to the perception of leader inclusiveness and expected no significant correlation between these measures.
Items of the self-report debriefing quality scale as well as respective sources
Trainee reactions
To assess trainees’ overall reactions to the training we used a German version of a scale measuring trainee's reactions to the training.3 This scale contained nine items which were rated on a 6-point Likert scale ranging from 1 (strongly disagree) to 6 (strongly agree) and were administered to the trainees after the last debriefing. Sample items are ‘The training was an effective use of my time’ and ‘The training was well-organised’. As the scale was originally published in English, the second author translated them into German and an independent interpreter back-translated it to ensure its validity.75 Cronbach's α was 0.87.
Psychological safety
Six items from the validated German translation76 of Edmondson's65 psychological safety scale—adapted to the clinical context—were administered before the first (ie, premeasure) and after the last debriefing (ie, postmeasure). Notably, this scale measured how safe the trainees felt taking an interpersonal risk within their usual teams, not during the training. Sample items are: ‘Everyone is able to bring up problems and tough issues’; ‘When someone makes a mistake it is always held against him’ (reverse coded to mitigate response set bias). One item of the scale ‘It is safe to take a risk in my team’ was excluded from the scale because it had significantly reduced the scale's internal consistency. Items were rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree); reliability analyses of trainees’ agreement resulted in moderate but acceptable Cronbach's α values of 0.69.
Leader inclusiveness
To measure the level of perceived leader inclusiveness before the first (ie, premeasure) and after the last debriefing (ie, postmeasure), we applied Nembhard and Edmondson's72 three item scale (eg, ‘My superiors ask for my ideas and opinions’) and adapted it slightly to fit the anaesthesia team training context. Again, items were rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree); reliability analyses resulted in Cronbach's α values of 0.73. Again, the second author translated them into German and an independent interpreter back-translated it to ensure its validity.75
Results
Trainees’ mean age was 39 years for the senior anaesthesia physicians, 31 years for the anaesthesia residents and 36 years for the anaesthesia nurses. Mean years of work experience was 10 years for the senior anaesthetists, 2 years for the anaesthesia residents and 7.5 years for the anaesthesia nurses (table 4).
Overview of trainees’ demographics
Validity of self-report debriefing quality scale
Table 5 shows the descriptive statistics and intercorrelations among the study variables. To test for the debriefing quality scale's convergent validity we investigated how it related to more general training perceptions, that is, the trainee reaction scale. We correlated the four measurement points of the debriefing quality scale with the trainee reaction scale. We found that the four measurement points of the debriefing quality scale positively correlated with the trainee reaction scale (all r>0.48, p<0.01; table 5).
Means, SDs and intercorrelations among study variables
To test for the debriefing quality scale's discriminant validity, we correlated the four measurement points of this scale with the scales measuring psychological safety and leader inclusiveness. Premeasured leader inclusiveness significantly correlated with the four measurement points (all r>0.31, p<0.05), but there were no significant correlations with premeasured and postmeasured psychological safety as well as with postmeasured leader inclusiveness (all r<0.17, p>0.23; table 5).
Evaluation of TeamGAINS debriefings
Trainees evaluated the debriefings as highly positive (M=4.22, SD=0.43) overall as during all four points of measurement (table 5). We found no significant relationships between age, sex, years of work experience, job role (ie, doctor vs. nurse) and debriefing quality (all r<0.19, p>0.17).
With respect to psychological safety and leader inclusiveness, we assumed that they would increase after the debriefings. We computed two two-sided paired t-tests. The results indicate that psychological safety significantly increased from t1 (M=3.36, SD=0.63) to t2 (M=3.48, SD=0.54); t(59)=−2.26, p=0.028 (95% CI −0.22 to −0.01). Likewise, leader inclusiveness increased from t1 (M=3.21, SD=0.68) to t2 (M=3.33, SD=0.56); t(60)=−2.07, p=0.048 (95% CI −0.23 to −0.003).
Discussion
The purpose of the study was to develop a debriefing structure and framework that integrates three established debriefing techniques and to evaluate this integrated debriefing approach with respect to debriefing quality and potential changes in team psychological safety and leader inclusiveness. We developed TeamGAINS, a hybrid, structured debriefing tool for simulation-based team trainings in healthcare that integrates guided team self-correction35 and advocacy-inquiry19 both being state-of-the-art debriefing methods, as well as systemic-constructivist techniques36 being the least prominent but most promising approach for exploring dynamic behaviour patterns within the team. By integrating these three different techniques into TeamGAINS and by providing guidance as to when and how to use them we aimed at combining their genuine advantages to exploit the full debriefing potential for exploring relationships between causes and effects of team behaviour during simulations.
We were interested in how useful trainees perceived debriefings conducted with TeamGAINS. We used TeamGAINS for debriefings during 10 one-day simulation-based training sessions of clinical and behavioural skills for anaesthesia teams. To evaluate the debriefing, we asked trainees to complete a self-report debriefing quality scale based on the DASH26 and the OSAD33 after each debriefing. We found that TeamGAINS was very positively evaluated. We also found that these evaluations did not differ with respect to job role, work experience, gender and age of the trainees. This indicates that TeamGAINS could be useful for debriefing doctors as well as nurses with very different levels of experience. We were also interested whether psychological safety and leader inclusiveness—two relevant team-related attitudes—would change over the course of one training day with four simulated cases and respective TeamGAINS-based debriefings. We found that the perception of psychological safety increased significantly, as did the perception of leader inclusiveness. More research is necessary to systematically study the impact of the steps and components of TeamGAINS on the debriefing process itself and on trainees’ changes in attitudes and clinical and behavioural skills. Before we point to limitations of our study and suggest future research ideas, we discuss the contributions of our approach to the science of team training.
Contributions to the science of team training
We think that providing TeamGAINS as a hybrid debriefing tool, demonstrating its perceived usefulness when used by experienced instructors and indicating the increase in psychological safety and leader inclusiveness during trainings in which TeamGAINS was used, is a promising contribution to research and practice of simulation-based trainings. This contribution is threefold.
First, we think that TeamGAINS contributes to using simulation as a team-training tool. Combining the advantages of guided team self-correction,35 advocacy-inquiry19 and systemic-constructivist techniques36 helps to use the teams’ simulation experience to identify, explore and change task- relevant interactions. Based on solid conceptual backgrounds, TeamGAINS offers guidance for simulation instructors with respect to (A) what to train, (B) how to elicit and voice positive and negative performance feedback in a non-threatening but honest way and (C) how to ‘walk the talk’ of teamwork by explicitly relating team performance gaps during simulation to team interactions and shaping new team interactions to close these performance gaps.
Second, we have demonstrated that it is possible and fruitful to apply concepts from systemic family therapy to train teams via simulation. Systemic-constructivist techniques offer possibilities to surface and change behaviour as a dynamic, interactive phenomenon of mutual influence among individuals rather than as a characteristic supposedly intrinsic to the individual.41 ,42 These possibilities had so far been unexplored in the science of team training.
Third, TeamGAINS contributes to the training of action teams, that is teams of highly skilled specialists who work together for brief performance events requiring flexibility and improvisation in an unpredictable context, often under high time pressure and with unstable team membership.77 As this team type is very prominent in healthcare (eg, trauma, anaesthesia, surgery) and also very risk-prone, it deserves more scientific attention and particularly tailored training.8 ,78 Rather than functioning via long-term planning or team building, action teams rely on adaptive coordination during brief performance sequences.79 ,80 TeamGAINS offers precise guidance for training this adaptive coordination.
Limitations and future research needs
It is noteworthy that our evaluation study has several limitations. First, we used a simple evaluation design and could not compare the effects of TeamGAINS with control group debriefings using any other debriefing methods. Second, all results rely on trainees’ self-reports. Third, we did not use the original debriefing assessment measures but used them as basis for a self-report debriefing quality scale. This scale had good reliability and reasonable content validity, as it was strongly based on the DASH26 and the OSAD,33 as well as reasonable convergent and discriminant validity, as it correlated with the scale measuring trainees’ overall reactions to the training but not with the scales measuring psychological safety and post-training leader inclusiveness. We therefore conclude that this scale served the purpose for measuring how learners received debriefings conducted via TeamGAINS. Still, our procedure involves limitations (eg, no reliance on behavioural anchors; reduced comparability with other studies) that could be prevented by using the original scales in future research. In addition, future debriefing studies could apply behaviour observation measures to assess debriefing quality, for example via videotaping and subsequent communication analysis (eg, developing an index for depth of reflection based on the number of elicited and changed frames and interaction patterns during a debriefing). This procedure would allow for investigating the immediate effects of each of the components of TeamGAINS, for example, trainees’ responses to suggestions from the Reflecting Team. It would also allow for comparing trainees’ responses to different TeamGAINS techniques, for example, to circular questions versus to inquiries following advocacies, and for testing for moderating effects of debriefer characteristics (eg, senior vs resident physician, clinician vs non-clinician). The incremental value of combining the three different debriefing techniques (eg, guided team self-correction technique only vs advocacy-inquiry technique only vs both techniques) should be tested via randomised control group designs. This would require the development of a conceptual debriefing model which would allow for deriving hypotheses on the optimal combined use of different debriefing techniques. We also think that more research is needed with respect to the long-term effects of debriefings based on TeamGAINS, for example via follow-up measures of clinical and behavioural skills.
In sum our study allows for concluding that debriefings which were conducted by experienced instructors according to TeamGAINS were well received by trainees and associated with increased psychological safety and leader inclusiveness. We consider this finding an important point of departure for the crucial next steps in detecting the enhanced effectiveness in debriefing that TeamGAINS was designed to promote. These next steps will involve (A) implementing a study design in which debriefing conducted with TeamGAINS is compared with debriefings conducted with other debriefing techniques, (B) developing measures which are sensitive to the detected differences between debriefings conducted via different techniques and (C) applying these measures during the analysis of videotaped debriefings.
Application of TeamGAINS
We think that more research and experience is required to learn what simulation instructors need to apply TeamGAINS. As TeamGAINS includes three different debriefing techniques we consider it rather complex, but so is conducting good debriefings. Our purpose for developing TeamGAINS was to offer guidance for simulation instructors by suggesting steps and questions but not to complicate their task. Learning to become a simulation instructor requires teaching, experience, reflection and deliberate practice.54 ,67 ,68 ,81 On their way to becoming advanced simulation instructors, instructors may begin with one of the debriefing techniques. With growing experience and confidence they may add more and more techniques to their debriefing repertoire. We fully acknowledge that this learning process needs to be studied and that we, as a next step, need to provide tools that meticulously facilitate this learning process during instructor courses and the use of TeamGAINS during training courses. For example, the ‘objective-oriented debriefing’37 and the most recent ‘scripted debriefing’28 are excellent examples of how to provide instructors with a tailored observation and communication guide based on the scenario's learning objectives. Having that in mind, as a first step we have created a figure for simulation instructors demonstrating TeamGAINS in a hopefully easy-to-follow way (see online supplementary figure 1). Still, TeamGAINS as such may for now be an ‘advanced debriefing technique’ for intermediate-level and advanced-level simulation instructors. Further research needs to provide insights into what simulation instructors need to incorporate TeamGAINS into their debriefing practice, and if there are any differences with respect to their professional background (eg, clinicians, psychologists) and how they can be addressed. In our study, TeamGAINS was applied by senior clinicians and expert psychologists—how it can be applied by other professionals with higher and lower levels of simulation-instructor experience remains to be shown.
Despite those significant future research needs, we believe that with TeamGAINS we have provided a promising tool for conducting structured debriefings during simulation-based trainings and for educating simulation instructors in the art and science of debriefing. We hope to have stimulated the use of TeamGAINS and the systematic comparison of its components’ effects on the debriefing process itself and on trainees’ changes in attitudes and in clinical and behavioural skills.
Acknowledgments
We would like to thank Adrian Marty, MD, for his support during the debriefings, Alfons Scherrer for his support during the trainings, and Burkhardt Seifert, PhD, for his statistical advice. We thank the anonymous reviewers for their very helpful feedback on earlier versions of this manuscript. We would also like to thank Jenny W Rudolph, PhD, and Kimberly Smith-Jentsch, PhD, for their very valuable comments on an earlier version of this manuscript.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online figure 1
- Data supplement 2 - Online table 1
Footnotes
-
Contributors MK developed the integrated debriefing approach and was responsible for the original study idea. MK, MW, GG and BG developed the design of the study. AK, MD and BG developed the simulated scenarios and organised the trainings. MK, MW, AK, MD and BG conducted the debriefings and collected the data. MW was responsible for data analysis. MK and MW drafted the manuscript and all authors contributed to its revision. GG and DRS were the principle investigator and co-investigator, respectively.
-
Funding This study was funded by a grant of the Swiss National Science Foundation (Grant Number 100 014_138545/1).
-
Competing interests None.
-
Ethics approval Ethics Committee of the Canton Zurich, Switzerland.
-
Provenance and peer review Not commissioned; externally peer reviewed.