Introduction

Anesthesiology has been at the forefront of high-fidelity simulation since its beginnings in the 1960s. Yet few studies identify the attributes that anesthesiology residents value in their educational experience with simulation.1 Therefore, we evaluated the various approaches to simulation-based education in Canadian anesthesiology programs with the aim of opening a dialogue between training programs on how best to further residents’ learning in simulated environments. Noting that “anxiety about performing in front of peers is the largest hurdle for anesthesiologists participating in simulation-based training exercises”, this survey identifies features of simulation training that foster a safe learning environment for residents and encourage successful learning experiences.2 The survey also compared residents’ perceptions of their training with the ten elements of a successful simulator exercise as outlined by the Best Evidence in Medical Education (BEME) review of simulation education.3

Methods

Ethics approval for this study was obtained from the University of British Columbia ethics review board.

Sampling frame

All 16 program directors at Canadian anesthesiology training centres were polled during a national meeting in the fall of 2007 regarding their program’s use of high-fidelity simulation. Consent to contact residents was obtained from each institution’s program director, and their respective secretaries were contacted via e-mail. Fifteen programs were identified that used high fidelity simulation as part of residency training. Questionnaire consent forms and an invitation to participate in the survey (with web link to the online survey) were e-mailed to program secretaries who distributed these documents to all residents on the departmental e-mail list. All 599 Canadian anesthesiology residents from the 1st to 5th years of post-graduate training who were listed on their university’s departmental e-mailing lists were included in the study. Residents in their first to third years of residency training were considered “junior residents”, and trainees in their final two years of residency training were considered “senior residents”.

The survey was electronically distributed at the completion of the 2007–2008 academic year (May 2008) and was made available for ten weeks. Two reminder e-mails were sent to residents during the 3rd and 6th weeks of survey availability online. The timing of survey distribution allowed residents to reflect on the past academic year’s simulation experiences.

Questionnaire development and piloting

A 56-question survey was developed to gather information from residents regarding their educational experience with high-fidelity simulation.4 Participants were invited to complete a ten-minute voluntary anonymous survey at: www.surveymonkey.com. All data was stored within the secured survey monkey website, which was downloaded to a departmental computer for analysis.

The questionnaire was rigorously designed and tested prior to administration, drawing on a comprehensive literature search and input from 25 senior Canadian anesthesiology residents and the lead author’s personal experience. The literature search was limited to human and English language articles. MEDLINE (1966 to May 2008), EMBASE (1980 to May 2008), The Cochrane Collaboration and The Campbell Collaboration libraries were searched using the terms “medical education”, “simulation”, “high fidelity simulation”, and permutations thereof. Hand searching references from papers collected and internet searching were also employed. To increase survey validity, drafts of the survey were piloted on 25 Canadian anesthesiology residents representing four training centres across Canada. Three iterations of the survey were reviewed by the pilot residents, and their feedback was integrated into the final version of the survey. To compensate for residents’ time and to increase study response rate, three gift certificate prizes were available for those who completed the online survey (total value $100).

Survey responses included Likert-scale multiple choice responses and open-ended questions. Mixed methods questions were also included in the survey, which combined both quantitative (scaled) responses with open-ended qualitative responses. A scaled rating methodology was chosen, as this was similar to previous studies that attempted to assess medical educational environments.5 , 6 Respondents were asked to indicate their agreement with each statement ranging from strongly disagree to strongly agree. Each response was given a numerical value of 0–4 depending on the response. Responses included: strongly disagree (0), disagree (1), neutral (2), agree (3), strongly agree (4). The survey can be found in Appendix 1 (available as Electronic Supplementary Material).

Question categories included demographics and simulator logistics, anxiety and learning climate, adherence to BEME principles, utility of simulation training, and the future of simulation training in anesthesiology. The questionnaire contained a number of dependent variables, including residents’ perceived anxiety compared with a usual day in the operating room (OR), their comfort in making mistakes in the simulator, as well as their comfort in asking for help during their simulation scenario.

Statistics

SPSS® 8 (SPSS Inc., Chicago, IL, USA) was utilized for statistical analysis. For certain multiple choice questions, raw data were presented as percentages of respondents “agreeing” or “disagreeing” with the statements (according to the Likert scale used).

Multiple choice questions are presented as median (interquartile range), while other multiple choice answers were dichotomized into two groups, i.e., those agreeing with the statement (including answers, “agree” and “strongly agree”) and those disagreeing (including answers, “disagree and “strongly disagree”). For open-ended responses, themes were found within responses and the “Top 5” responses were reported. Numerical data, such as the resident’s age, the number of training sessions in which the resident participated, and the number of supervisors per training session, are presented as means.

Results

Demographics

One hundred and sixty-seven Canadian anesthesiology residents completed the questionnaire. Responses by year of training, gender, and residency program are displayed in Figs. 1 and 2.

Fig. 1
figure 1

Questionnaire response rate for 15 Canadian residency training centres utilizing high fidelity simulation

Fig. 2
figure 2

Questionnaire response rate by gender and residency training level

Anxiety

Eighty-one percent of residents ranked their anxiety level during simulation training as either “higher” or “much higher” than during a regular day in the OR. Performing in front of colleagues and staff increased residents’ stress levels, with 78% stating that their anxiety was either “high” or “much higher” with these observers in attendance. Eighty-two percent of residents said they felt less anxious as they participated in more simulation scenarios (Table 1). Senior residents believed that their performance anxiety abated due to an increased familiarity with the team, being more comfortable making mistakes, and improved medical knowledge. This is in contrast to junior residents who believed they became less anxious with more experience due to increased familiarity with the equipment and “the set-up” of the simulated OR. Sixty-eight percent of all respondents thought that advance notice of the critical incident topic presented would decrease their level of anxiety, and 57% of all residents wanted their supervisor or a senior resident to complete the first scenario of the training session, understanding that they would then complete a simulated scenario independently.

Table 1 Simulator session frequency and anxiety responses according to year of training

Gender

There were notable differences between gender with respect to the level of anxiety in the simulator compared with a regular day in the OR, the comfort level making mistakes and asking for help in the simulator, and whether performance in the simulator should be used as part of formal yearly assessments. Gender data are presented in Table 2.

Table 2 Anxiety responses with respect to gender

Best evidence in medical education

Individual school responses with respect to BEME principles are reported in Table 3.

Table 3 Training centre specific responses with respect to BEME recommendations

Focus of simulation exercises

As high-fidelity simulation can replicate a wide variety of clinical situations, we were interested in the scenarios that were most frequently encountered by Canadian anesthesia residents in their training. Residents were asked to rank the most common simulation scenarios they had experienced during their training. The top five rankings for junior residents were: (1) advanced cardiac life support (ventricular tachycardia/bradycardia); (2) difficult airway; (3) anaphylaxis; (4) mechanical/machine problems/power loss; and (5) malignant hyperthermia. The top five rankings for senior residents were: (1) malignant hyperthermia; (2) obstetrical emergencies; (3) mechanical/machine problems/power loss; (4) increased airway pressures/ventilation difficulties/airway fire; and (5) Tie: advanced cardiac life support and difficult airway.

Future directions: utility of simulation training, introductory courses for junior residents and improvements

Ninety percent of residents responded that the use of patient simulators played an important role in their critical incident training as an anesthesiologist, and 88% indicated that simulator training improved their practice of safe anesthesia. Yet, 46% of residents responded that their performance in simulator sessions should not be used in their yearly assessments as residents.

Sixty-five percent of respondents thought that participation in simulation education should be required by staff anesthesiologists’ continuing medical education programs. Thirty-six percent of residents agreed that simulation scenarios will eventually become part of the Canadian Royal College examination in anesthesiology, while 37% remained neutral and 18% “disagreed”. There was no appreciable difference in response to this question with respect to training level.

Eighty-one percent of residents agreed that an introductory simulator course teaching how to manage common intraoperative emergencies should exist for junior residents. Fifty-one percent of respondents indicated that the ideal length of such a course should be “several days”, followed by 35% of respondents who indicated that a one-day course would be the ideal length.

Improvements to simulator training

Simulation is a powerful learning tool; however, as with any tool, the outcome is the measure of its effect on the learner. Learner feedback is an integral component of a successful adult learning experience; therefore, we were interested in residents’ perceptions as to how learning through simulation could be improved. Residents reported five recommendations to improve learning through simulation. From most commonly cited to least, the key recommendations included:

  1. 1.

    Increased access to simulation;

  2. 2.

    Improved reality of the simulated learning environment;

  3. 3.

    Increased participation in multidisciplinary scenarios;

  4. 4.

    More qualified supervisors/more interest from staff in simulation/diversity of supervisors; and

  5. 5.

    Use of an introductory course for junior residents addressing how to deal with common anesthetic emergencies.

Discussion

Current approaches to adult learning rely on learners assuming an active role in their own education. As noted by colleagues in orthopedic surgery, a fundamental principle guiding education should be, “What is learned is more important than what is taught”.7 Simulated learning environments should not be exempted from this principle. It is essential that simulation training be based on the best evidence and theory related to learning in such environments. Despite a somewhat limited response rate, the findings from this survey suggest the existence of a close relationship between residents’ perceptions of their learning and what we know about adult learning more generally.

Anxiety and performance

Anxiety about performing in front of peers is one of the largest hurdles for experienced anesthesiologists participating in simulation training.2 , 8 Although it is assumed that simulation training is a safe non-threatening environment, similar trepidation exists among anesthesiology residents participating in simulation exercises. Eighty-one percent of residents rank their anxiety level as “higher” or “much higher” when performing in front of colleagues and staff in a simulated environment than performing on a regular day in the OR with these observers in attendance. It appears that the anxiety of participants is increased by a combination of not knowing the scenario the resident will be asked to manage, performing in front of staff and colleagues, and limited previous experience with simulation. If educators wish to promote learning, it is important to realize the level of anxiety learners experience while in a simulation-based learning exercise.

The Yerkes-Dodson law suggests that an empirical relationship exists between the stress of an activity and individual performance.9 It claims that for any given task there is an optimal level of cognitive arousal for peak performance, with too much (overload) or too little (boredom) arousal resulting in sub-optimal performance. Specific to anesthesiology, stress in the OR may cause the anesthesiologist to have tunnel vision, use chaotic scanning patterns, limit the capacity of working memory, and cause a disparity between the speed at which actions are performed and the accuracy with which they are performed.Footnote 1 These errors can all have disastrous consequences for patients.

Only 43% of residents felt comfortable making mistakes in a simulator environment compared with 81% of residents who felt comfortable asking for help. If residents are given a chance to ask for help before a critical error has occurred, it appears that they are comfortable in doing so. However, once a mistake has been made, the residents feel uncomfortable in the simulator. It was interesting to note that male residents were more likely than female residents to feel comfortable making mistakes and asking for help in the simulator (Table 2). This observation highlights the importance of individualizing each simulation scenario and debriefing techniques according to each individual learner’s needs and personalities.

Ninety-one percent of all residents reported they were “rarely” or “never” given the opportunity to call timeouts during simulation exercises. Timeouts could help bridge the time from when residents start to feel uncomfortable in a scenario to when they begin to move to a sub-optimal position on the Yerkes-Dodson plot. Introducing “timeouts” into simulation exercises would also facilitate a “reflection in action” approach rather than a “reflection on action” approach to debriefing.10 An important tenet of a reflection in action approach is that residents are given the opportunity to repeat a problem by employing a different strategy. Once the students have identified a learning problem by calling a timeout, it allows them to determine their needs in relation to performing a particular clinical scenario. The method of reflection in action not only gives other residents participating in the scenario an opportunity to understand the thought process and situational awareness of the resident “in charge”, but it also allows for ongoing feedback for the resident rather than debriefing taking place at the completion of the simulation exercise.

Best evidence in medical education

Issenberg et al. recently completed a Best Evidence in Medical Education (BEME) review that determined features of high-fidelity simulation training that lead to effective learning.3 A 34-year period was reviewed and 109 journal articles were identified that examined simulation as an educational intervention or assessment. Several key components were identified, including providing feedback, providing the opportunity for repetitive practice, integrating simulation into the educational curriculum, providing a range of difficulty, employing multiple learning strategies, offering clinical variation, working within a controlled environment, making individualization of learning an option, having defined learning outcomes, and ensuring simulator validity. The majority of Canadian simulation training programs appear to follow the ten recommendations outlined in the BEME paper (Table 3 for scenarios) and to have a close regard for the frequency and quality of feedback given during simulation exercise.

Debriefing and feedback on performance were identified by Issenberg et al. as the most important components for learning during simulation exercises. In our study, 57% of all residents received feedback from their supervisors, 16% from other residents, and 27% from both parties and other participants in the scenario. Of note, 60% of the residents surveyed noted that videotaping was “very” useful to “critically” useful as part of their simulation training. However, it was also noted that 80% of debriefing sessions were “instructor-led”. Perhaps increasing residents’ involvement in feedback exercises would help to alleviate some of the anxiety of performing in front of supervisors.

The components of effective feedback have been previously outlined.Footnote 2 Specifically, effective feedback should be for the trainee’s benefit, should be given in a manageable amount, and should be in close proximity to the observed behaviours. Also, feedback should be offered, not imposed, and should be specific. These are the characteristics of effective feedback that were assessed in the questionnaire. Overall, 92% of residents received their feedback immediately after their scenario session was completed. Ninety-four percent of residents thought that feedback was provided in manageable amounts and was not overwhelming. Seventy-eight percent of the residents noted that specific examples of their performance were used during feedback, and 96% believed that the feedback was constructive and provided for their benefit.

Learning theory postulates that learning a new skill requires repeated practice, yet 68% of the residents were not allowed to repeat the parts of the scenario where they had experienced difficulty, and 81% were not given the opportunity to repeat the entire scenario even when they chose to do so. If time constraints were a contributing factor, perhaps scenarios should be shorter and/or less complex to make time available for repetitive deliberate (focused) practice. Indeed, 50% of residents surveyed believed that scenarios should be shorter with more focused learning objectives.

The importance of adult learners having input into the direction of their learning cannot be overlooked. Fifty-five percent of residents felt simulation sessions were not “tailored to their learning needs”. Additionally, 57% of residents noted that learning objectives were “often” or “sometimes” stated during their simulation exercises. A short discussion with trainees the week prior to entering the simulator may allow learning objectives to be more tailored to the specific group of residents.

Future directions

Canadian anesthesiology residents have embraced high-fidelity simulation as a key modality in their education, with an overwhelming majority claiming that simulator training improves their management of critical incidents and enhances their practice of safe anesthesia. However, finding ways to introduce more simulation training into residency curriculum is clearly a challenge. The use of introductory simulation courses early in residency may be an answer. As noted in our study, introductory simulation courses for junior residents are not in widespread use in Canadian post-graduate anesthesiology programs, but perhaps they should be. In Scotland, an introductory course exists for new anesthetic trainees that provides an excellent example of how simulator courses can be integrated into a residency curriculum.11

Another future direction involves the true definition of the term “high fidelity”. True high-fidelity simulation in the OR should not only strive to emulate the environment, equipment, and resources of the real OR, but it should also endeavour to represent the mix of health care professionals working in this environment. Our survey suggests that 76% of residents “rarely” or “never” practice simulation with individuals from other departments or disciplines, such as nurses or residents from different specialties. An increase in multi-disciplinary learning in the simulator could prepare residents to work within multi-disciplinary teams and would familiarize OR staff with one another. It could also improve the crisis resource management skills of the entire OR team. Next to increased access to high-fidelity simulation, the residents’ most popular responses regarding improvements that could be made to the current anesthesiology simulation program were a more representative simulated environment and the use of more multidisciplinary scenarios.

It is likely that simulation training will become part of the certification examination for Canadian anesthesiology residents in the near future. The Tel Aviv University in Israel already employs simulation as part of the board certification examination for anesthesiologists.12 For reasons of patient safety and our imperative of first, do no harm, simulation should also play a central role in the re-certification and continuing medical education of anesthesiologists in Canada. As we move in this direction, our motto should be: What can be learned outside the OR, should be learned outside the OR.

Limitations

In addition to the inherent bias of questionnaires, the highest proportion (20.9%) of responses came from the authors’ training centre. This simulation training centre may not necessarily be representative of a typical Canadian simulation training centre and may have introduced significant sampling bias. Given the overall response rate of 27.9%, and although multiple attempts were made to contact non-responders, the issue of non-responder bias may be inherent to this study. However, in keeping with our overall response rate, a previous online survey of resident doctors in Canada had a similar response rate of 27.4%, and a recent review of 199 online surveys from a variety of disciplines indicated a comparable average response rate of 32.5%.13,14

Given the enormous variety of uses for simulation in anesthesiology and the limited access to this valuable training modality, it is vital to understand the importance of organizing and structuring individual simulation sessions to maximize student learning and retention. This study provides insight into the experience of Canadian anesthesiology residents as they re-create reality in the OR.