SIMULATION FOR TRAINING IS EFFECTIVE WHEN …

There is no question that simulation can be an effective tool for training complex skills. There is some evidence that it works.¹ But it is only a tool. As with any tool, in order to be effective it must be used appropriately. We commend the paper by Satish and Streufert² in this issue of QSHC for highlighting the role that simulation may play in both training and assessment within the medical community, as well as the recognition that effective simulation must: (1) be built on underlying theory (they use complexity theory), (2) use structured exercises, and (3) assess performance and provide feedback. However, some additional observations about simulation are warranted so that scientists and training developers within the medical community do not fall into some common myths and misconceptions known to exist regarding training in general, as well as the use of simulation for training.³ We therefore present a few observations based on the science of training^1,4 and our experience in aviation and military environments about when simulation is effective for training.^5,6 Simulation for training is effective when …

(1) … instructional features are embedded within the simulation

Simulations to facilitate learning need to be designed around key instructional components—that is, simulation based training must have a series of links that create a learning environment (fig 1). One instructional strategy that has been successfully used in aviation and military environments and embeds the above instructional features is the event based approach to training (EBAT). This strategy relies on the “a priori” embedding of multiple events into the scenario at different time intervals. These events serve as cues for trainees to exhibit competencies targeted in training.⁷ In turn, these cues serve as measurement and feedback opportunities. Advantages to this approach include:

• ensuring opportunities to exhibit targeted behavior are presented;

• scenario control while giving the appearance of a free flowing scenario;

• increasing the ease with which competencies can be measured;

• providing standardization across trainees.

Simulation can therefore only create opportunities for learning if instructional features are built into it.

(2) … carefully crafted scenarios are embedded within the simulation

Satish and Streufert² suggest that the SMS simulation can define scenarios “a priori”. However, further clarification is needed regarding the factors that drive scenario parameters. One must remember that, in simulation based training, the scenario(s) are the curriculum so they must be carefully storyboarded. This could be facilitated by performing a cognitive task analysis (CTA). A CTA should help in determining the content of the scenarios since it will uncover the cues expected to be used to perform complex tasks. In addition, scenarios should build events into scripts. These inserted events serve as “triggers” and provide known opportunities to both practise and assess important behaviors. Scenarios are therefore a key component for simulation to facilitate learning and cannot be left to chance or created without a learning outcome in mind.

(3) … the simulation contains opportunities for assessing and diagnosing individual or team performance

As noted above, we agree with Satish and Streufert² that simulation based training will only be effective to the extent that trainee competence can be assessed. There are two points to this statement.

Firstly, simulation based training must provide measurement opportunities that ease the burden on those responsible for measurement. More specifically, simulations that use pre-scripted learner focused scenario events not only ensure that relevant competencies are being assessed, but ease the assessment process as instructors know when key events will occur.

Secondly, not only does simulation need to build in opportunities for the assessment of performance, but also these measurement opportunities must provide the basis for diagnosing skill deficiencies. In other words, it is not enough that the simulation provides opportunities to capture performance outcomes, but it must also (as much as possible) capture the moment-to-moment actions and behaviors. These process oriented measurements are much richer for training purposes; they are also the most difficult to capture and may require human intervention. Simulations that include measurement systems which only capture outcome measures (such as quality or quantity) do not allow those responsible for training to diagnose performance; they do not offer information on how to improve performance. Performance measurement is paramount to training and, without it, simulations are just that—simulations.

(4) … the learning experience is guided

We have all heard the saying “practice makes perfect”. Similarly, it has been argued that experience—that is, practice—can make an excellent teacher because it generates knowledge within a meaningful context.² The conditional nature of both these statements needs to be highlighted—that is, practice or experience in and of itself does not equal learning. Trainees who are given unguided practice often:

• learn the wrong thing;

• do not focus on practising the right behaviors;

• may spend too much time on only one particular aspect of training;

• may not be able to transfer the skills to the job.

To maximize the learning experience, practice must be guided (through carefully crafted scenarios and diagnostic timely feedback) so that trainees remain focused on learning key competencies.

(5) … simulation fidelity is matched to training requirements

When using simulations for training purposes it is often assumed that more is better; this is not true. For example, research has found that use of high physical fidelity simulations in training did not transfer or had very little effect on actual job tasks.⁸ Similarly, research has successfully used low fidelity PC based simulations to train complex individual and teamwork skills.^9–12 The level of simulation fidelity needed should be driven by the cognitive and behavioral requirements of the task and the level needed to support learning.⁵

Finally, simulation for training is effective when …

(6) … there is a reciprocal partnership between subject matter experts and learning/training specialists

Learning is a behavioral/cognitive event. Training is about imparting longlasting change in trainees. It is about creating a context where key competitiveness can be practised, assessed, diagnosed, remedied, and reinforced. To do that requires a partnership between task experts and those who know about the design and delivery of training. No one can do it effectively alone. Both parties have something to contribute: subject matter experts articulate task requirements and needs while training specialists create learning environments. Both are needed and the medical community should foster it.

CONCLUSION

Simulation is an effective tool for training complex skills. The military and aviation environments have invested heavily in simulation based training and, although further multilevel assessments need to be conducted, initial data regarding its effectiveness are encouraging.^13–15 However, simulation is only a tool, and training developers and practitioners must rely on the science of training to maximize the effectiveness of it. There are known principles. Our recommendation is to apply them, and to develop a partnership with those who understand what it takes to design and deliver effective training.

Simulations must be designed so that: (1) instructional features are embedded within the simulation, (2) carefully crafted scenarios are embedded that contain opportunities for performance measurement and diagnostic feedback, (3) the learning experience is guided, and (4) simulation fidelity is matched to task requirements. Keeping all this in mind, the medical community can gain great benefits from using simulation to train both individuals and teams to reduce human error and promote patient safety.