Background: During the evaluation of many instances of the same basic surgical skill, we observed that there were several errors that occurred frequently. Two studies were undertaken to examine the use of these errors for improving the instruction and evaluation of the skill.
Materials and methods: For both studies, two types of rater training videotapes were developed. One involved the use of examples of common errors (error) and the other demonstrated the skill being performed correctly (correct). A testing videotape was created consisting of 24 performances of the skill that ranged in quality of the performance. The first study was designed to assess the impact of error instruction on skill acquisition. In this study, a group of 30 senior medical students were randomly assigned to one of four different training groups: none, error only, correct only, and error+correct. Subjects were videotaped performing the skill before and after the training and three experts evaluated these performances independently using a 7-point rating scale. The second study was designed to assess the impact of error training on skill evaluation and was done using both novice and expert raters. The same group of 30 senior medical students used in the first study was used as novice raters. Following training in one of the four training groups, each subject rated the 24 performances on the testing videotape and interrater reliability was assessed for each group. Surgical faculty served as expert raters in this study and were randomly assigned to receive either error training or no training. Each subject viewed the testing videotape, rating the performances and giving "feedback" commentary. Interrater reliability was calculated for the two groups and the precision of the feedback was assessed.
Results: Significant improvement in posttest performance scores was seen only in the "error+correct" training group. Interrater reliability was somewhat lower for the "correct only" and "error only" training groups in both the student and faculty studies. Faculty raters receiving error training had a higher proportion of specific comments than the group that received no training although this difference was not statistically significant.
Conclusions: Instruction about common errors, when combined with instruction about the correct performance enhanced the acquisition of this surgical skill. This suggests a role for the use of errors in surgical technical skill instruction. Our study provides no support for a role for error training in improving skill evaluation.