Article Text
PDF
Abstract
Objectives The authors conducted a randomised controlled trial of four pedagogical methods commonly used to deliver teamwork training and measured the effects of each method on the acquisition of student teamwork knowledge, skills, and attitudes.
Methods The authors recruited 203 senior nursing students and 235 fourth-year medical students (total N=438) from two major universities for a 1-day interdisciplinary teamwork training course. All participants received a didactic lecture and then were randomly assigned to one of four educational methods: didactic (control), audience response didactic, role play and human patient simulation. Student performance was assessed for teamwork attitudes, knowledge and skills using: (a) a 36-item teamwork attitudes instrument (CHIRP), (b) a 12-item teamwork knowledge test, (c) a 10-item standardised patient (SP) evaluation of student teamwork skills performance and (d) a 20-item modification of items from the Mayo High Performance Teamwork Scale (MHPTS).
Results All four cohorts demonstrated an improvement in attitudes (F1,370=48.7, p=0.001) and knowledge (F1,353=87.3, p=0.001) pre- to post-test. No educational modality appeared superior for attitude (F3,370=0.325, p=0.808) or knowledge (F3,353=0.382, p=0.766) acquisition. No modality demonstrated a significant change in teamwork skills (F3,18=2.12, p=0.134).
Conclusions Each of the four modalities demonstrated significantly improved teamwork knowledge and attitudes, but no modality was demonstrated to be superior. Institutions should feel free to utilise educational modalities, which are best supported by their resources to deliver interdisciplinary teamwork training.
- Communication
- medical education
- teamwork training
- culture
- randomised controlled trial
Statistics from Altmetric.com
Individual and group cognitive processes1 and teamwork interactions2 ,3 all contribute to the safety of healthcare environments. Human-factors engineers in military systems have developed effective teamwork training curricula for military applications, successfully converting high-risk organisations into high-reliability ones.4–6 To develop such education for healthcare, the Agency for Healthcare Research and Quality (AHRQ) funded the development of the TeamSTEPPS Patient Safety Program, derived from training materials from US Army Aviation, the US Department of Defense, the Institute of Medicine,7 ,8 Tactical Team Decision-Making under Stress9 and MedTeams.10 ,11 The TeamSTEPPS programme is designed to improve care and patient safety through improved teamwork training to enhance provider communication skills. While the 7 h multidisciplinary TeamSTEPPS programme has proven very effective, questions remain regarding the most effective and cost-efficient approach to providing this content to learners at all stages of the health-education continuum.
Various educational approaches have been used to address training in team coordination skills in healthcare. Methods such as crisis resource management training using high fidelity human patient simulation provide realism and interactivity.1–3 However, this training modality is both costly and time-consuming, with one instructor per five trainees costing as much as $5000 per day.12 Because of this, medical and nursing schools are implementing more traditional didactic approaches to teaching team coordination skills as well as computer-based instruction, small-group sessions, analysis of videotaped team coordination scenarios and workplace practice experiences.10 ,11 ,13 Questions remain about the effectiveness of any of these methods to achieve the necessary changes in student attitudes, knowledge and skills, and there is little evidence for selecting particular methods. Therefore, this study was designed and implemented to adapt the TeamSTEPPS content to pre-licensure nursing and medical students, and measure the effectiveness of four educational interventions at teaching this material.
Methods
Protocol
We conducted a full-day teamwork training and assessment exercise for 438 students composed of fourth-year medical students (N=235) and final-semester nursing students (N=203) from four health-professional schools at two major universities: all students convened on the one campus where they were randomised to one of four educational intervention groups. Students were offered the option of not participating in the research project but could not decline the educational experience. Institutional review boards of both universities approved the project.
The day-long pre-postintervention study design followed the protocol in figure 1. All participants attended a 90 min didactic lecture providing an overview and framework of teamwork training. The lecture focused on three core components of the TeamSTEPPS curriculum including: Situational Awareness, Shared Mental Model and Leadership. The content highlighted the utility of Briefs, Call-outs, Check-back and De-briefs for given patient encounters, and the advantages of using SBAR and CUS words as tools for communication among team members. Students then separated into assigned groups to participate in one of four instructional modalities: (a) high-fidelity human-patient simulation (n=80 equally divided by discipline); (b) role play (n=80, equally divided by discipline); (c) audience response system lecture (n=140); and (d) traditional didactic lecture (n=138). All four of the instructional modalities provided an additional 60 min of teamwork training. Participants were rerandomised by cohort for team-based standardised patient (SP) assessments, most of which were videotaped for later scoring. To measure changes in student teamwork knowledge and attitudes, written knowledge and attitudes instruments were given as a pre- and post-test at the beginning and end of the day. Both medicine and nursing faculties were trained to lead the simulations and role plays in a 2 h faculty development session focused on facilitation, core TeamSTEPPS content and the case studies.
Research evaluation design.
Cohort A: high-fidelity human patient simulation
Teams composed of four students, two medical and two nursing, participated in two interactive patient care scenarios based on core concepts of the didactic training. A combination of 10 METI ECS and Laerdal SimMan human simulators operated simultaneously so that 80 students completed the simulation exercises in a 2 h period. Each simulation exercise included preprogrammed scenarios to create a realistic patient interaction, with a faculty facilitator who played a ‘confederate’ role in the scenario. Sessions were video-recorded using the B-Line Clinical Skills System to allow immediate debriefing with the faculty facilitator. Although clinical actions and outcomes were addressed, debriefing focused on team cooperation and behaviours encouraged in the didactic lecture.
Cohort B: low-fidelity simulation role play
Over a 2 h period, 10 concurrent groups of role play participants (N=79) worked in four-person teams of two medical and two nursing students with the same two scenarios used in the high-fidelity simulation. Scenarios were paper-based and performed as a true role play with cards and visual prompts but no mannequin. The faculty facilitator helped progress the scenario by distributing cue cards with patient information to individual team members. Participants in Cohort A and B wore disposable lab coats labelled for their role as a charge or floor nurse, and as a medical intern or chief resident to enhance the realism of the environment and to identify team members. After each role play, the faculty facilitator debriefed participants on team behaviours as above.
Cohort C: audience-response didactic
One large group of participants (N=140) attended a lecture that included videotaped scenarios from the TeamSTEPPS curriculum and, while differing from simulation materials, emphasised the same teamwork skills. An Audience Response System (ARS) was used to capture and display the participants' anonymous responses for group discussion. Group discussion focused on key points to enhance student understanding of key teamwork behaviours.
Cohort D: didactic lecture alone
Participants remained in a large auditorium (N=138) to watch the same lecture slides and TeamSTEPPS video as Cohort C participants, but without the ARS interactive slides. The faculty presenter answered any questions by students but did not initiate questions or facilitate ongoing dialogue about the material being covered. This cohort served as the control group for the study.
Standardised patient teamwork skills assessments
For the day's last activity, students were randomised into 110 new four-member teams from within their original cohort. Each group, of two nursing and two medical students, participated in a 20 min standardised patient (SP) exercise designed to elicit teamwork skills. Each participant was provided essential information needed by the team in order to treat the patient successfully and do a handoff from Nurse A/Doctor A to Nurse B/Doctor B. Participants shared their information with the team, completed a focused patient history and physical exam, and completed orders for the patient. The SP scored the team using a 10-item behaviourally anchored checklist. All participating SPs completed a 6 h training session with case specifics and scoring metrics. SPs were blinded to the student identity and training modality. Using the B-Line Clinical Skills System, 88 SP sessions were videotaped, representing a random sample of the four cohorts.
Measures
Four instruments were developed to measure teamwork knowledge, skills and attitudes: a 36-item CHIRP-Teamwork Attitudes instrument, a 12-item Teamwork Knowledge test (multiple versions), a 10-item case specific Standardised Patient Evaluation (SPE) of teamwork skills performance, and a 20-item modification of the Mayo High Performance Teamwork Scale (MHPTS).14 The knowledge and CHIRP attitudes instruments were completed by participants, and the SPE instrument was completed by the standardised patient upon case conclusion. The modified High Performance Teamwork Scale was completed by an independent scorer while viewing the videotaped scenarios.
We originally used the first eight items of the published, 16-item Mayo HPT Scale (MHTPS),13 with two items split into four items to bring the total to 10 items. A pre-hoc analysis of 20 randomly selected videotapes were scored on these preliminary 10 items by seven trained raters. Based upon interrater reliabilities (range 0.360 to 0.670 on seven items with the remaining three items at −0.665, 0.063 and 0.182), the scale was modified to 20 items. A different set of 32 randomly selected videotapes (stratified across educational intervention cohorts) were scored on the revised 20 items by four new, trained raters to assess reliabilities, with results described below.
Analyses
All analyses were conducted using SPSS, Version 16.0 (SPSS, Chicago, Illinois). For the pretest–post-test comparisons on the Attitudes and Knowledge tests, repeated-measures Generalised Linear Model (GLM) analyses of variance (ANOVA) were conducted to compare all four cohorts (Simulation, Role play, Audience Response Simulation, Lecture) or comparisons of composite groups (eg, Simulation combined with Role play (Interactive) vs Audience Response combined with Lecture (Didactic) comparisons). These were planned comparisons based upon the Research Design (figure 1).
For teamwork skills, two evaluation methods were used: a SP Patient Checklist and an independently scored Mayo HPT scale. For the SP Evaluation, the standardised patients evaluated each of the 110 afternoon teams using the 10-item Standardised Patient Checklist for Teamwork Skills (table 1). Complete data were available for 68 teams across the four cohorts, and given the number of comparisons 64 error degrees of freedom were used in the SPSS one-way ANOVA.
Sample items from the four measurement scales
For the 20-item revised version of the Mayo HPT Scale (MHTPS) instrument, independent raters scored 32 teams evenly divided across the four cohorts. On this one-way ANOVA, complete data were available on 22 teams, and given the number of comparisons 18 error degrees of freedom were used.
Results
The sample included 429 consenting students, and analyses were based on up to 370 sets of complete data (86.2%), depending on the measurement instrument. Because of the pre–post comparison design, any surveys that could not be paired with the companion instrument (pre and post) were excluded.
Participants' attitudes towards teamwork improved significantly from pre- to post-test (F3,370=48.7, p=0.001) in all four cohorts (table 2a). When compared by educational methods, there were no significant differences between cohorts (F3,370=0.325, p=0.808). Additionally, there were no significant differences between the interactive and didactic groups (F1,372=0.068, p=0.794), between the simulation and role play cohorts (F1,126=0.779, p=0.379) and between the ARS and lecture only (control group) cohorts (F1,244=0.273, p=0.602) (table 2a). The mean scores and SEs for these comparisons are provided in table 2b.
Pre-to-posttest ANOVA comparison of teamwork attitudes across cohorts and subcohorts
Participant scores on the knowledge post-test were significantly higher than pre-test scores (10.16 vs 9.13; F3,353=87.3, p<0.001) (table 3a); mean scores and SEs per cohort are provided in table 3b. However, there were no significant differences between the interactive and didactic groups (F1,355=0.200, p=0.655), between the simulation and role play cohorts (F1,93=0.006, p=0.938) and between the Audience Response System (ARS) and lecture only (control group) cohorts (F1,260=1.08, p=0.300) (table 3a). The mean scores and SEs for these comparisons are provided in table 2b.
Pre-to-posttest ANOVA comparison of teamwork knowledge across cohorts and subcohorts
The SP evaluation of teamwork skills showed excellent inter-rater reliability between clinician raters and standardised patient scores, ranging from 0.683 to 0.968; therefore the standardised patient scores were used. There were no significant differences between the four cohorts (68 teams across the four cohorts) in SP ratings of teamwork skills on the 10-item scale or for non-parametric item-by-item comparisons (table 4).
Total score comparisons for standardised patient evaluation (10 items)
The revised, 20-item Mayo High Performance Teamwork Scale demonstrated inter-rater reliabilities with Interclass Correlation Coefficients (ICC) ranging from 0.83 to 1.0, on 19 of the 20 items. There were no differences between the four cohorts (eight teams per cohort for a total of 32 teams assessed) using one-way ANOVA (F3,18=2.12, p=0.134) (table 5). Examination across raters showed no significant differences on any of the 20 checklist items between the four cohorts. Sample items from all instruments used are demonstrated in table 1.
Cohort comparisons for revised Mayo High Performance Teamwork Scale (20 items)
Discussion
Healthcare professions students must have the tools and methods to improve teamwork skills if they are to become safe providers in our ever-more complex healthcare system. The TeamSTEPPS curriculum was designed to provide this training, and our results demonstrate that even brief teamwork training using the TeamSTEPPs curriculum significantly improves student attitudes and knowledge about team skills. The lack of substantial differences between educational methods suggests that, at least for basic teamwork training, any fidelity of training environment or educational pedagogy can provide substantial increases in basic learner competency in core teamwork knowledge and attitudes. This is important because it liberates educators and institutions from the belief that effective training in basic teamwork knowledge, attitudes and skills is only possible with expensive interventions or high-fidelity simulations.
Beaubien and Baker15 have identified three principal-components of simulation fidelity: equipment, environment and psychological fidelity. They argue that the most applicable component of teamwork training, psychological fidelity, can be achieved in low-technology simulation environments.15 ,16 Our study shows that neither interactive lecture nor low- or high-fidelity simulation is more effective in improving basic teamwork training in inexperienced healthcare providers than traditional didactic education alone. These results indicate that any of the four options tested are viable methods to teach teamwork concepts. One should not interpret these results as reducing the potential role of simulation, but they do raise questions about whether simulation is necessary to achieve measurable results, particularly for basic team training. Institutions with limited resources for creating high-fidelity simulation environments should not be deterred from implementing teamwork training via any method.
Our study was enriched by its interdisciplinary cohort of students and faculty from two disciplines and two universities. Faculty facilitators reported comments from student participants that they learnt first hand about each other's roles and training. Opportunities to clarify misperceptions and begin to form trusting relationships can help establish an environment of mutual respect before students work together in the clinical arena. This knowledge may assist in mitigating poor working relationships between providers which are a known factor in nurse turnover and the growing national nursing shortage.17 Although we cannot demonstrate this with our results, our anecdotal feedback suggests that our students' awareness of the importance of other members of the healthcare team was enhanced by the experience of learning together.
Our study focused on teaching students a process for a structured approach to teamwork communication skills using methods such as SBAR and CUS words as well as structured interventions such as Briefs, Call-outs, Check-back, and De-briefs. There are limited data to assist educators in the design and the allocation of curriculum time and resources required to teach these skills to senior level physician and nursing students. Studies of the doctor patient relationship have demonstrated that complex perceptual skills require years and perhaps a lifetime to master, while process skills can be taught and learnt in brief interventions.18 Our study has demonstrated that the basic structured methods of teamwork communication can be taught and learnt using focused educational interventions comprising any of the four pedagogical methods examined in this investigation. We did not evaluate the retention of these skills or, more importantly, assess their application in real-world clinical settings; however, we have demonstrated that the critical first steps of awareness and basic knowledge can be achieved even with brief time-limited interventions.
The design of the study may have limited its ability to discern significant differences in the effectiveness of the educational interventions. Participants were randomised into cohorts without considering individual learning styles or preferences, which could have flattened the results. In addition, because this was a new deployment of this content for all schools, we did not have a working baseline from which to judge the level of content. We attempted to assess this with the pre–post design however; our preknowledge scores were higher than those predicted by faculty who designed the instrument. This may have represented a level of content expertise that we did not predict given that none of the four participating schools formally taught this content in their curriculum, or it may have represented a failure of the instrument to discriminate between learners. Second, a lack of TeamSTEPPS specific instruments limited our confidence in scoring team behaviours. In addition, many participants had no previous experience with standardised patients or high-fidelity simulation prior to the training, and this could have hindered the expression of teamwork skills, which more experienced simulation learners may have demonstrated. Further, the time allotted for students to participate in the simulated scenarios (30 min per scenario, 1 h total) was limited, and this may have negatively affected the educational effect of the interactive experiences. Finally, the learners were not observed longitudinally, and questions remain about the impact of various pedagogies in changing long-term behaviours in clinical situations. Subsequent measures of skill retention some days or weeks after the intervention would have strengthened our results.
Conclusions
This is the largest study to date comparing four pedagogical methods of teamwork training, a critical component of modern healthcare curricula. All the modalities appeared effective in changing knowledge and attitudes, and providing this education in a multidisciplinary manner produced a rich educational milieu for both the learners and educators. Institutions interested in team training should not feel limited to a specific fidelity of training environments; rather, medical and nursing schools should use this material in an integrated curriculum to achieve critical change in students' capacity to form effective healthcare teams. Longitudinal studies are needed to evaluate the most effective pedagogies for changing behaviour in the clinical setting.
Acknowledgments
We gratefully acknowledge the support of METI, Inc. (Sarasota, Florida) and Laerdal Medical, Inc. (Wappingers Falls, New York), who provided simulators for the project, and B-Line Medical, LLC (Washington, DC) who provided technical support for performance monitoring.
References
Footnotes
The Interprofessional Patient Safety and Education Collaborative is an interdisciplinary, interinstitutional team focused on improving patient safety and team training education. Its members are: B Foster, B Goldstein, C Durham, C Mayer, C Grochowski, D Hollar, A Felix, G Sherwood, C Hobgood, D Woodyard, H Love, J Taekman, P Kinneer, K Frush, K Turner, L Cronenwett, M Holtschneider, M Wright, N Segall, C Osmond, V Kaprielian, S Sawning, L Maynard and S Promes.
Funding The Galaxo Smithkline Foundation Research Triangle Park Durham, North Carolina.
Competing interests None.
Ethics approval Ethics approval was provided by the UNC Health affairs IRB and Duke University Health Affairs IRB.
Provenance and peer review Not commissioned; externally peer reviewed.