The use of simulation in medical education is increasing in frequency and in sophistication. Perhaps the most advanced simulation tool yet constructed is a computer-controlled anthropometric manikin developed by the University of Southern California, a device now known popularly as “Sim One”. This paper reports the results of an experiment testing the effectiveness of Sim One in the training of anesthesiology residents in the maneuver of endotracheal intubation.

Sim One is lifelike in appearance, having a plastic skin which resembles that of a real human being in color and texture; its configuration is that of a patient lying on an operating table, left arm extended and ready for intravenous injection, right arm fitted with blood pressure cuff, and chest wall having a stethoscope taped over the approximate location of the heart. It breathes; has a heart beat, temporal and carotid pulse (all synchronized), and blood pressure; opens and closes its mouth; blinks its eyes; and responds to four intravenously administered drugs and two gases (oxygen and nitrous oxide) administered through mask or tube. The physiologic responses to what is done to him are in real time and occur “automatically” as part of a computer program.

The entire maneuver of endotracheal intubation can be performed in exactly the manner in which it is done in the operating room. At the outset, oxygen is administered through mask-and-bag, following which there is an injection of sodium pentothal. Approximately 20 seconds later, Sim One’s eyes close and unconsciousness is simulated. Succinyl choline is injected, bringing about complete relaxation of muscles and cessation of breathing. Through the use of a laryngoscope, a tube is inserted into the trachea and oxygen and nitrous oxide are then administered by pressure on the reservoir bag.

The assumption of several educational “advantages” led to the statement of hypothesis underlying this study. The first potential advantage is that the use of the simulator allows for a planned and gradual increase in the difficulty of the problems to be solved by the anesthesiology resident instead of his performing new tasks in the operating room as the necessity for them arises. Second, the use of the simulator permits almost unlimited repetition of any phase of the procedures to be learned. Third, the resident can obtain immediate feedback on his own performance in the form of a printout of all acts performed in the order in which they took place and with the exact time elapsed also noted. Finally, each learner proceeds at his own rate.

It was hypothesized that residents trained on the simulator would achieve predesignated criterion levels of performance in less time and with fewer operating room trials than residents not permitted to use the simulator.

STUDY DESIGN

The anesthesiology resident training program at the Los Angeles County General Hospital is one of the largest in the country: at any given time there are more than 24 residents in training. It was anticipated that 12 new residents would arrive after 1 July 1967. These individuals were to be paired for purpose of study, one of each pair to be afforded training on the simulator and the other to engage in the usual training procedures without the benefit of experience on the simulator. The decision as to which member of each pair would be given training on the simulator was made by a flip of a coin. The pairing was accomplished over a period of time as each resident arrived (between the late spring of 1967 and the fall of 1967). Eventually, only five pairs were used because two of the residents had had extensive experience in endotracheal intubation during the year prior to beginning their residency training, each having accomplished more than 150 intubations. The five residents assigned to Sim One completed their training on the simulator in varying periods of time, ranging from 5.5 hours to 9.5 hours over a 2 week period. Had the visits to Sim One not required a considerable amount of cross-city travel, each resident could have completed the exercise in 2 or 3 days.

The design of the study necessitated some measure of proficiency in endotracheal intubation. The plan was to observe each of the 10 residents in the operating room every time he was required to participate in endotracheal intubation. In addition, similar observation was to be made of the five simulator-trained residents during their training runs on Sim One. Comparisons between simulator-trained residents and those in the control group were to be made on the basis of elapsed time from date of arrival in the program to date of performance at a professional level of proficiency.

The investigators used as the source of data the official anesthesia charts of the hospital. Each anesthesiologist is obliged to maintain a complete record of the administration of anesthesia during surgery procedures as part of the official hospital records. These anesthesia charts include a sufficient amount of information concerning his proficiency.

The anesthesia charts of all endotracheal intubations performed by the 10 residents in the study were subjected to critical appraisal. The names of the resident, the patient, and other personnel, as well as the date of the intubation were completely concealed and each chart was submitted to the Department of Anesthesiology with this question posed: “On the basis of what you see on this chart, would you be willing to trust the anesthesiology resident in an operating room without supervision?” The raters gave the charts a “plus” rating for professionally acceptable performance and a “minus” rating for unacceptable performance. In all, 1220 charts were reviewed and rated.

Following completion of the rating process, the anesthesia charts were sorted according to residents’ names and arranged in chronological order from the date of the first intubation completed by the resident to that of the last intubation within the time period of the study. Three criterion levels of performance were selected: the number of operating room trials necessary for a resident to achieve 4, 7 of 8, and 9 of 10 consecutive plus ratings. As a second variable the investigators examined the number of elapsed days between the first endotracheal intubation in the operating room and achievement of each of these three criterion levels.

FINDINGS

The data were subjected to analysis of variance (F ratio) using a randomized block design for all analyses. Despite the small number of subjects, statistical significance was tested. The results are presented in table 1 which lists the mean number of trials and days necessary for the residents to achieve each of the criterion levels of performance and are summarized in fig 1.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

 The graph on the left shows the mean number of trials and the one on the right the mean number of elapsed days required by five experimental and five control subjects to reach three prescribed criterion levels of performance.

The smallest differences to be noted are in the number of elapsed days and number of trials required to achieve four consecutive plus ratings. As the more rigorous criteria are applied (7 of 8 and 9 of 10), the differences tend to be larger and a 0.01 level of statistical significance is reached for the last performance criterion (9 of 10 consecutive plus ratings). Furthermore, the difference between the mean number of elapsed days required to meet this criterion is significant at the 0.05 level.

DISCUSSION

Despite the fact that four of the six analyses failed to reach a level of statistical significance, the investigators conclude that there is a time advantage in using this computer-controlled patient simulator in the training of anesthesiology residents. The results of the study are all in the hypothesized direction—that is, residents using the simulator reached accepted professional levels of performance in fewer elapsed days and in a smaller number of trials in the operating room than did residents who did not have a training period on the simulator.

The failure to reach statistical significance can be accounted for by the large within-group variability and the small number of subjects involved in the study. The differences in means for each of the criterion levels are substantial, if not significant. But the figures, perhaps, do not tell the whole story. For instance, a minus rating signifies that some aspect or aspects of total performance were so lacking in quality that an experienced anesthesiologist stated that he would not trust this man in the operating room without supervision. Thus, potential discomfort or even harm is posed to the patient during that endotracheal intubation. Significantly less threat to patient welfare is posed by residents who have been trained on the patient simulator. The mean number of trials necessary for simulator-trained anesthesiology residents to achieve four consecutive ratings is 9.6 as compared to 18.6 for those not trained on the simulator so, on average, nine more patients were posed with minor, or less minor, threat by each beginning anesthesiology resident not trained on the simulator before that resident achieved skill enough to perform four consecutive professionally acceptable endotracheal intubations. These differences grow as the criterion level becomes more exacting.

In addition, Sim One promises a significant time saving in training. Again, despite the lack of statistical significance in two analyses, the differences between experimental and control groups in the time required to reach each criterion level are all in the hypothesized direction; and the one significant result demonstrates a 22 day time saving, on average, in the achievement of 9 of 10 consecutive plus ratings—the most exacting criterion applied. Thus, by extrapolation, beginning anesthesiology residents might be expected to achieve this level of professional competence in a saving of 22 days over a period of 77 days. If all of the skills to be learned by anesthesiology residents could be taught through simulators, one can speculate that the achievement of these skills might be accomplished in less than three quarters of the time now required.