Article Text
PDF
Abstract
Context Several studies show that communication errors in healthcare teams are frequent and can lead to adverse events. Team training has been suggested as a way to safer communication and has been implemented in healthcare as classroom-based or simulation-based team training or a combination of both. The objective of this paper is to systematically review studies evaluating the outcomes of classroom-based multiprofessional team training for hospital staff.
Method The authors searched PubMed, EMBASE, ERIC, PsycInfo, Cinahl and the Cochrane Reviews database and selected 18 studies for description and comparison of learners and setting, objective, design, intervention, evaluation methods (reaction, learning, behaviour and results), intervention time before evaluation, outcomes and risk of bias.
Results Participant reactions were positive. Learning and behaviour were positive in all studies, but for some only partially. The effect on clinical processes was in most instances positive. Results at patient level were limited. Only one study reported results at all four evaluation levels. Fifteen studies were uncontrolled, and 17 studies had a moderate or high risk of bias. More than half of the studies ended evaluation within 6 months. No studies reported qualitative measures that could have provided an insight as to why the interventions had the effect they had.
Conclusion Classroom-based team training for multiprofessional hospital staff is recommended as a way to improve patient safety. This review shows mainly positive effects of the intervention on participant reaction, learning and behaviour. The results at clinical level are still very limited.
- Team training
- crew resource management
- patient safety
- communication
- outcomes
Statistics from Altmetric.com
Introduction
Poor teamwork and verbal communication between healthcare staff have been found to be correlated with adverse events, staff performance problems and higher patient morbidity and mortality.1 There could be several reasons for this problem; some of the most influential might be differences between staff groups2 and a complex work environment.3 Team training is recommended as a method to improve communication and coordination in high-reliability organisations.4 ,5 Team training for healthcare staff came on the agenda after IOM's ‘To err is human,’ and a critical analysis suggested that the medical field introduced Crew Resource Management (CRM) as one of 79 practices to reduce the number of adverse events.6 ,7 Increased specialisation, more acute and complicated procedures and shorter hospital stays call for more communication in shorter time. Institutions advocating safety in healthcare now recommend hospitals to introduce communication tools8 ,9 or team training.10 ,11
Team training has been transferred to healthcare as classroom-based or simulation-based team training or a combination of both. Simulation is an educational technique that allows realistic interaction by recreating a clinical experience without exposing patients to the associated risks.11 ,12 This is often accomplished through the use of mannequins and advanced software.13 Classroom-based interventions uses lectures, video demonstrations, discussions and role plays11 ,14 ,15 to strengthen participants teamwork, communication and coordination knowledge, skills and attitudes. For organisations aiming at training larger groups of staff members the classroom-based model is tempting, as it allows many to train at one time at lower costs than the equipment- and instructor-demanding simulation-based method. The question is, however, whether this type of training is effective. The objective of this paper is to systematically review studies evaluating the outcomes of classroom-based team training for multiprofessional hospital staff.
Methodology
Literature search
The following sources were searched for results of classroom-based team training interventions for multiprofessional hospital staff published in peer-reviewed journals through March 2010: PubMed (including MeSH), EMBASE, ERIC, PsycInfo, Cinahl and the Cochrane Reviews-database (figure 1 shows the combination of search terms).
Literature search and study selection process.
The following MeSH-terms were used: ‘Patient Care Team’, ‘Interdisciplinary Communication’ and ‘Outcome Assessment.’ Articles in the following languages were considered: English, German, French, Italian and the Scandinavian languages. A ‘hand search’ was conducted by reviewing the reference lists of relevant articles. Eligible articles included in the review described classroom-based team/non-technical skills/crew resource management training interventions focused on communication and coordination training using didactical and interactive methods to improve the participants' knowledge, skills and attitudes of teamwork skills and the clinical outcome. Articles alone referring the development or implementation of programmes, pregraduate programmes, extra-hospital, web-based, mono-disciplinary, patient or relative-centred or mainly simulator-based interventions were excluded. Articles describing the effect of brief instructions before the use of preoperative briefing checklists were excluded, as the instruction was not considered training.
Data extraction and analysis
The selected studies were reviewed with focus on the following parameters: ‘Learners,’ ‘Setting,’ ‘Programme,’ ‘Objective’ and ‘Design.’ The ‘Intervention’ was reviewed for duration, methods and contents of the course,16 extent of a needs assessment17 and how training transfer was supported (table 1).40 We specifically analysed the ‘Evaluation and level of evaluation’ based on Kirkpatrick and Freeth: (1) What was the participants' reaction to the course? (2) What did they learn? (knowledge (2a), skills (2b) and attitudes (2c)). (3) Did training make individuals change behaviour? (4) What results were obtained regarding wider change in clinical processes? (4a) and clinical outcomes (4b)?41 ,42 It should be noted that self-rated behaviour was categorised as ‘learning’ of skills whereas observed behaviour and other more objective data (including self-reporting from patients) were categorised as behaviour or results respectively.43 ‘Time from intervention to evaluation’ and ‘Risk of bias’ based on (1) study design (controlled/uncontrolled; randomised/not randomised; prospective/retrospective), (2) loss of participants to follow-up and (3) blinding of observers was also reviewed. Based on this assessment, we assigned each study a quality rating: ‘High’ (high risk of bias), ‘Moderate’ (moderate risk of bias) and ‘Low’ (low risk of bias)44 (table 2).
Studies evaluating classroom-based multiprofessional team training for hospital staff. Outline of setting, objective, design and intervention from published studies
Evaluation methods and outcomes of studies evaluating class-room based multiprofessional team training in hospitals
Results
Out of 4236 citations studied, a total of 18 studies18–21 ,24 ,26–30 32–39 met the inclusion criteria. All studies were published in English. One study was Swiss,24 one was Australian,37 and two were British.28 ,38 The rest were American. One study was a cluster randomised controlled trial.30 Two were prospective controlled.29 ,39 The rest were prospective uncontrolled,18 ,21 ,24 ,26 ,28 ,32–38 retrospective controlled,19 retrospective uncontrolled20 or a case study.27 The learners were multiprofessional hospital staff members. The objectives by and large focused on evaluating the outcomes. All studies except two19 ,36 described a process of training needs assessment, the main method being a safety or teamwork attitude questionnaire (SAQ/TAQ), use of patient safety data and inputs from staff. The duration of the course varied from 4 h to 3 days (a few also described longer train-the-trainer courses).26 ,31 ,37 ,39 All interventions focused on teamwork, coordination and communication.
Six studies reported participant reactions, and all described very positive responses.21 ,24 ,34 ,36 Fourteen studies evaluated the effect on learning18 ,19 ,21 ,24 ,26 ,28 ,29 ,32–35 37–39: All studies used before–after SAQ or TAQ and reported positive outcomes on some or most items. However, one subgroup analysis revealed a significantly improved score for one of two intervention sites but not for the other.28 Another study found significantly improved scores for surgeons and anaesthetists but not for nurses.18 A third found perceived benefit of briefings higher among nurses than among anaesthetists and surgeons.26 Two studies assessed knowledge: one found a significant increase from before to after.37 Another found high overall knowledge after.39
Nine studies evaluated the effect of the intervention at the individual observed behavioural level.18 ,20 ,26–29 ,35 ,37 ,39 Behavioural change was measured through the use of perioperative briefings and was mostly positive: in one study, compliance increased from 0% to 86% after training, but decreased to 66% after 6 months.26 Another found 64% compliance after 1 month but 100% compliance after four. Another reported significantly more briefings but no absolute numbers.39 Three studies measured behaviour as use of communicative frameworks and found improved teamwork scores.29 ,37 ,39 One study found teams compliant with 60% of the recommended practices after a year (after brief retraining).20 Another only found an increase in team non-technical skills at one of two intervention sites.28 Behaviour was in yet another study reported as staffs' increasing willingness to report incidents.35
Seven studies evaluated the effect on process measures: four found improvement,18 ,29 ,34 ,37 two found partial improvement,28 31 and one found no improvement.26
Four studies reported outcome measures at the patient level: two found no effect on patient satisfaction29 and on an Adverse Outcome Index, AOI (defined as the percentage of women who experience one or more of a number of prespecified adverse events).31 However, two studies found an improvement in patient safety through a significant reduction in AOI.32 ,33
There was a tendency towards a positive effect of a local multiprofessional work group conducting or participating in an intense follow-up-phase after the intervention.23 ,32 ,33 ,37 However, the descriptions of follow-up in the studies (and their related published curriculum descriptions) are limited.
Discussion
This review shows that the field of classroom-based team training is still new with few published studies and limited proof of clinical results. However, participants overall reacted positively to training and improved their knowledge and attitudes. The participants in most instances improved professional behaviour, and most studies of process measures showed an improvement. As such—before describing the reservations to these results—at least we know that the concept is well received by hospital staff. This is an important primary indicator for the intervention in healthcare.
One relevant Cochrane review was identified.45 However, this 2008 review contained only six studies, and only one of these was relevant for this review.29 The Cochrane review concludes that the small number and the heterogeneity of studies make it impossible to generalise on the clinical effect of interprofessional education, and more rigorous research is needed.
This is possible due to the substantial challenges for this kind of intervention:
First, except for three studies,29 ,30 ,39 the studies (N=15) had very weak designs. The uncontrolled before–after studies have a great risk of unwanted time-related effects on the outcome of interest: staffing problems, patient issues and change in the economic situation of the unit or hospital. Controlled designs are preferred, but standardisation can be hard in the complex settings. Triangulation (use of both qualitative and quantitative measures) and methods such as statistical process control can strengthen the before–after design.46 ,47
Second, as readers, we still do not know much why an intervention was effective and another less effective, as the studies often were brief on descriptions of needs assessment, planning, training and follow-up. The internet gives authors the option of presenting (and sharing) course curricula, follow-up plans, questionnaires and observation tools as e-appendixes (as done in a few cases).11 ,28 ,36 Further, we found no reports of qualitative measures as interviews with staff focussing on why the intervention had the effect it had. Such measures could contribute to a deeper insight and should be encouraged.
Third, in most cases, the evaluation took place at only one, two or three levels. This is too limited to provide the reasoning that is the rationale for the many evaluation levels: in order to render demonstrated clinical results probable presentation of outcomes at behaviour, learning and reaction levels are necessary. This evaluation burden is significant but can be reduced if sharing is encouraged. Evaluations by outside observers and other more objective data are also important, as experiences show a tendency to over-reporting in self-rating of behaviour.43
Fourth, more than half of the studies were evaluated within 6 months. For interventions aiming at improvement in clinical outcomes, this is too soon: Experiences from other fields show that it takes a sustained effort and thorough follow-up after training for a new teamwork culture to root in the organisation.48–50 This includes structural changes, changes in policies and procedures, retraining, training of new staff members, support of practise, role modelling, feedback and development of well-functioning checklists.
Further research is necessary before giving the intervention a general recommendation.
Limitations
We included the studies after a thorough search of relevant, mainly medical, databases, but other educational, sociological and psychological databases may contain relevant references.
At the same time, the terminology is imprecise and changing (for instance, the terms ‘team training’/‘crew resource management’ (as used in mainly American literature) and ‘non-technical skills training’ (as used in the British literature) are somewhat synonymous). This leads to heterogeneous indexing in bibliographic databases. To compensate for this, we conducted a thorough hand search. However, the result of the search might still be incomplete.
Our categorisation of the evaluation parameters into the four evaluation levels might be faulty, especially with regard to ‘behaviour,’ ‘process measure’ and ‘outcome measure.’ It is based on often brief descriptions. The aim was to standardise the often varying categorisation in the papers, not to devaluate the results achieved.
Conclusion
Classroom-based team training for multiprofessional hospital staff is recommended as a way to improve patient safety. This review shows mainly positive effects of the intervention on participant reaction, learning and behaviour. The results at the clinical level are still very limited.
References
Footnotes
Funding The Pharmacy Foundation of 1991 and Det Kommunale Momsfond funded this study.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.