Abstract
The introduction of laparoscopic techniques to general surgery was associated with many unnecessary complications, which led to the development of skills laboratories to train novice laparoscopic surgeons. This article reviews the tools currently available for training and assessment in laparoscopic surgery.
Medline searches were performed to identify articles with combinations of the following key words: laparoscopy, training, curriculum, virtual reality and assessment. Further articles were obtained by manually searching the reference lists of identified papers.
Current training involves the use of box trainers with either innate models or animal tissues; it lacks objective assessment of skill acquisition. Virtual reality simulators have the ability to teach laparoscopic psychomotor skills, and objective assessment is now possible using dexterity-based and video analysis systems.
The tools are now available for the development of a structured, competency-based, laparoscopic surgical training programme.
Introduction
The first reported laparoscopic cholecystectomy was performed by Phillipe Mouret in France in 19871 and within 5 years the laparoscopic approach was established as a feasible alternative to open cholecystectomy2–8. Patients experienced the benefits of smaller incisions, a shorter hospital stay and decreased postoperative pain, and could resume normal activities within a week2,9–13. Initial reports of success, the enthusiasm of many surgeons and an increasingly competitive healthcare market led to a large number of operators attempting the new technique. However, doubts soon surfaced regarding its safety and the qualifications of those performing the procedures14–19. This prompted the surgical community to reconsider the training strategy in laparoscopic surgery.
Societies and regulatory bodies such as the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) and the European Association of Endoscopic Surgeons (EAES) stipulated minimum requirements for those performing laparoscopic surgery, with an emphasis on training both in and outside the operating theatre20–24. Skills courses and drills were introduced to teach basic psychomotor skills. As the number of surgeons performing the procedure increased, more opportunities were available for novice surgeons to assist. Laparoscopic cholecystectomy gradually became a safe and effective alternative to the open procedure for a diseased gallbladder2,25–27.
The advent of laparoscopic general surgery
The skills required to perform laparoscopic surgery are different to those of open surgery, being more allied to endoscopy than to traditional laparotomy. The surgeon has to enter the peritoneal cavity using a smaller incision, use long instruments with only their tips visible and become accustomed to the fulcrum effect28. Procedures are performed by viewing a two-dimensional video image on a screen up to 2 m away, with limited tactile feedback29–31.
Traditional methods of acquiring surgical skill, using the apprenticeship model, could not accommodate the new skills required for laparoscopic surgery. This was the first time a completely new technique had been introduced and few surgeons had seen a laparoscopic cholecystectomy, let alone performed one. Experienced biliary surgeons assumed that their skills at open surgery would transfer to the laparoscopic environment, but they too were novices in the field32. As Krummell33 has written, it was necessary for the surgical curriculum to evolve away from the age-old apprenticeship model and toward the teaching of skills in a systematic and logical fashion by doing, rather than through observation. The surgical community realized that the early phase of the learning curve could be achieved outside the operating theatre and that once proficiency had been demonstrated the surgeon could proceed to real operations.
Laparoscopic training outside the operating theatre
The introduction of laparoscopic courses for general surgeons was an attempt to teach basic laparoscopic skills within a structured curriculum. Courses lasted for 2–3 days and consisted of didactic lectures with some hands-on simulator practice. Synthetic models enabled surgeons to practise tasks in box trainers and varied from performing simple tasks with different laparoscopic instruments to the learning of entire procedures34–37. Many courses involved performing a laparoscopic cholecystectomy on a cadaveric porcine model, and some on an anaesthetized pig38–40.
Surgeons were able to acquire skills in a controlled environment, free of the pressures of operating on real patients. This enabled the acquisition of basic skills for proficiency in laparoscopic surgery with emphasis on the fact that attendance at such courses was not to be interpreted as a licence to perform unsupervised laparoscopic procedures41. The success of this form of training is now evident from the large number of laparoscopic courses available worldwide. However, box trainers have been criticized for being unrealistic and lacking any form of objective assessment. Computer simulators have been used to train airline pilots for a number of years, providing realistic simulation with an accurate assessment of performance. The possibility of an analogous situation in surgery is becoming a reality.
Virtual reality systems in laparoscopic surgery
The term virtual reality refers to ‘a computer-generated representation of an environment that allows sensory interaction, thus giving the impression of actually being present’42. The MIST-VR® laparoscopic simulator (Mentice, Gothenburg, Sweden) comprises two standard laparoscopic instruments held together on a frame with position-sensing gimbals43 (Fig. 1). These are linked to a Pentium® personal computer (Intel, Santa Clara, CA, USA) and movements of the instruments are relayed in real time to a computer monitor. Targets appear randomly on the screen and are ‘grasped’ or ‘manipulated’ (Fig. 2), with performance measured by time, error rate and economy of movement for each hand.
The Minimally Invasive Surgical Trainer—Virtual Reality (MIST-VR®)
Sample tasks on the MIST-VR® laparoscopic trainer. a acquire place; b traversal; c manipulate diathermy; d stretch diathermy
The LapSim® (Surgical Science, Gothenburg, Sweden) laparoscopic trainer has tasks that are more realistic than those of the MIST-VR®, involving structures that are deformable and may bleed44 (Fig. 3). The Xitact LS500® (Xitact, Morges, Switzerland) laparoscopy simulator comprises tasks such as dissection, clip application and tissue separation, the integration of which can produce a procedural trainer45. It differs from the MIST-VR® and LapSim® in that it incorporates a physical object, the ‘virtual abdomen’, with force feedback. Other newer simulators include the Reachin Laparoscopic Trainer® (Reachin, Stockholm, Sweden), ProMIS™ Surgical Simulator (Haptica, Dublin, Ireland) and LapMentor™ (Simbionix, Cleveland, Ohio, USA).
Sample tasks on the LapSim® laparoscopic trainer. a cutting; b suturing
The MIST-VR® simulator has tasks that are abstract in nature, enabling the acquisition of psychomotor skill rather than cognitive knowledge. This enables the simulator to be used in a multidisciplinary manner to teach the basic skills required for all forms of minimally invasive surgery. However, newer simulators have augmented their basic skills programmes to incorporate parts of real procedures, allowing trainees to learn techniques they would use in the operating theatre. For example, the LapSim® has a module for dissection of Calot's triangle, and the recently launched LapMentor™ simulator enables the trainee to perform a complete laparoscopic cholecystectomy with the benefit of force feedback. Although the task-based simulators are more advanced in terms of software, they are bulkier and more expensive.
Using these simulators, trainees can practise standardized laparoscopic tasks repeatedly, with instant objective feedback of performance. The simulators are portable, use standard computer equipment and are available commercially. With graded exercises at different skill levels, they can be used as the basis for a structured training programme. The feedback obtained also enables comparisons to be made between training sessions and trainees.
Learning curves on laparoscopic trainers
Studies to assess the ability of box and virtual reality trainers to teach laparoscopic skills have analysed the learning curves of experts and novices. Smith et al.46 constructed a box trainer containing posts of varying heights and ten non-surgeons touched each post in a sequential order, a total of ten times. The laparoscopic instruments were connected to sensors recording their position in space, hence accuracy of movement was a measure of performance in addition to time. Subjects improved their time taken during the first three repetitions, although accuracy of movement continued to improve throughout the ten sessions. This implies that the learning curve for operator speed is shorter than that for operator accuracy.
Gallagher and Satava47 reported data on experienced, inexperienced and novice laparoscopic surgeons who completed ten trials of six tasks on the MIST-VR®. The first trial revealed significant differences between the three groups, the experienced group being the fastest and most consistent. Performance variables for all groups had reached a plateau by trial 5, with most improvement for novice and inexperienced groups. With repeated practice on the simulator, the novices had improved their performance to match that of experienced surgeons.
A criticism of laparoscopic trainers is that they are abstract, with tasks being too simple and not related to real procedures. Rosser et al.48 described a structured training method for enhancing laparoscopic surgical skills, with 150 trainee surgeons performing a series of three simple standardized laparoscopic drills on a box simulator, ten times each. At the end of each drill their skill at performing an intracorporeal stitch was assessed by time taken. After only one series of tasks, the time taken to complete an intracorporeal stitch improved significantly, with further significant improvement at the end of ten repetitions. An equivalent programme with MIST-VR® as the training tool revealed significant improvements in knot-tying times after a 5-day training period49.
Transfer of skill from laparoscopic trainer to human patient
Before the incorporation of simulators into the general surgical curriculum, it is necessary to show transfer of laparoscopic skill to real operations. Scott et al.50 assessed 22 general surgical trainees performing a laparoscopic cholecystectomy on a human patient. Nine were enrolled in a 10-day training programme on a laparoscopic box trainer; the remainder underwent no formal training. The trained group improved to a significantly greater extent than the control group on four of the eight assessment criteria, with non-significant improvements on the remaining criteria.
The first study to assess the transfer of skills from a virtual reality simulator recruited 16 surgical trainees of varying experience (postgraduate years 1–4), who were randomized to training on the MIST-VR® simulator or to a control group51. The simulator group trained on MIST-VR® until they attained a previously specified expert score. All subjects then completed a human laparoscopic cholecystectomy, scored by two independent observers using a checklist for gallbladder dissection. The trained group dissected the gallbladder 29 per cent faster; they were nine times more likely to make progress and five times less likely to make errors. However, there was no evaluation of baseline laparoscopic skill before the study, merely a non-specific test of psychomotor ability. A subsequent study of surgical skill on a real laparoscopic cholecystectomy recruited 16 laparoscopic novices (fewer than ten procedures performed previously) and divided them into a MIST-VR®-trained group and a control group52. The trained group performed significantly faster than the control group and had greater improvement in error and economy of movement scores.
Although performed with small numbers of subjects, these initial studies have shown transfer of laparoscopic skills from virtual reality systems to real operations. This may be a more cost-effective method of imparting surgical skill, although it is necessary to make comparisons with current methods of training, namely box trainers.
Comparison of standard and virtual laparoscopic trainers
Hamilton et al.53 randomized surgical trainees to ten half-hour sessions on a box trainer or to the MIST-VR®, with baseline and post-training skill assessments on both trainers. All achieved significant improvements regardless of which simulator they had trained on, although the magnitude of improvement for the MIST-VR®-trained group was significantly greater than that for the box-trained group on both forms of assessment.
However, evidence for the role of virtual reality simulators in the transfer of skills to the operating theatre remains weak. There is a lack of standardization in the operative techniques used and assessment is performed by direct observation, enabling the observer to be influenced by attitudes of the surgical team to the operating surgeon. Time alone has been shown to be a poor indicator. The development of objective measures of operative skill is important to confirm the role of simulators in laparoscopic surgery, and if successful may lead to further advances in the credentialing and revalidation of all surgeons.
Assessment of technical skills in laparoscopic surgery
In 1991, SAGES required surgeons to demonstrate competency before performing a laparoscopic procedure21,23,24. Competency was based on the number of procedures performed and time taken, or on evaluation of the trainee by senior surgeons30,54–56. These criteria are known to be crude and indirect measures of technical skill, or to suffer from the influence of subjectivity and bias. Professional organizations have recently recognized the need to assess surgical performance objectively57. Currently, this consists of dexterity analysis and video-based assessment, which also enables structured progression during training, together with identification of trainees who require remedial action.
For any method of skill assessment to be used with confidence, it must be reliable, valid and feasible. Reliability is a measure of the precision of a test and supposes that results for a test repeated on two separate occasions, with no learning between the two tests, will be identical. It is measured as a ratio from 0 to 1·0, a test with reliability of 0–0·5 being of little use, 0·5–0·8 being moderately reliable, and over 0·8 being the most useful. Validity refers to the concept of whether a test measures what it purports to measure. Face validity refers to whether the model resembles the task it is based upon, and content validity considers the extent to which the model measures surgical skill and not simply anatomical knowledge. Construct validity is a test of whether the model can differentiate between different levels of experience. Concurrent validity compares the test to the current ‘gold standard’, and predictive validity determines whether the test corresponds to actual performance in the operating theatre.
Dexterity analysis in laparoscopic surgery
Laparoscopic surgery lends itself particularly well to motion analysis, as hand movements are confined to the limited movements of the instruments. Smith et al.46 connected laparoscopic forceps to sensors to map their position in space, and relayed movements of the instruments to a personal computer. This enabled calculation of the instrument's total path length, which was compared to the minimum path length required to complete the task.
The Imperial College Surgical Assessment Device (ICSAD) has sensors placed on the back of a surgeon's hands58,59. A commercially available device (Isotrack II™; Polhemus, Vermont, USA) emits electromagnetic waves to track the position of the sensors in x, y and z axes 20 times per second (Fig. 4). This device is able to run from a standard laptop computer and data are analysed in terms of time taken, distance travelled and total number of movements for each hand. Previous studies have confirmed the construct validity of the ICSAD as a surgical assessment device for laparoscopic procedures, both for simple tasks60 and for real procedures such as a laparoscopic cholecystectomy61. Experienced laparoscopic surgeons made significantly fewer movements than occasional laparoscopists, who in turn were better than novices in the field. The ICSAD device has also been shown objectively to assess the acquisition of psychomotor skill of trainees attending laparoscopic training courses62,63.
The Imperial College Surgical Assessment Device (ICSAD). a standard laptop and motion tracking hardware; b sensors placed on surgeon's hands for assessment of laparoscopic skill on a box trainer
The Advanced Dundee Endoscopic Psychomotor Tester (ADEPT) is another computer-controlled device, consisting of a static dome enclosing a defined workspace, with two standard laparoscopic graspers mounted on a gimble mechanism64. Within the dome is a target plate containing innate tasks, overlaid by a spring-mounted perspex sheet with apertures of varying shapes and sizes. A standard laparoscope relays the image to a video monitor. Each task involves manipulation of the top plate with one instrument enabling the other instrument to negotiate the task on the back plate through the access hole. The system registers time taken, successful task completion, angular path length and instrument error score (a measure of instrument contact with the sides of the front plate holes). Experienced surgeons exhibit significantly lower instrument error rates than trainees on the ADEPT system65. Comparison of performance on ADEPT also correlated well with a blinded assessment of clinical competence, a measure of concurrent validity66. Test–retest reliability of the system produced positive correlations for all variables when performance on two consecutive test sessions was compared67.
These three methods of assessing dexterity enable objective assessment of surgical technical skill, but only the ICSAD device can be used to assess real operations. However, in this case it is important to know whether the movements made are purposeful. For example, the common bile duct may be injured during a laparoscopic cholecystectomy, and dexterity analysis alone cannot record this potentially disastrous error. To confirm surgical proficiency it is necessary to analyse the context in which these movements are made.
Video-based assessment in laparoscopic surgery
During the introduction of laparoscopic cholecystectomy, SAGES and EAES advocated proctoring of beginners by senior surgeons before awarding privileges in laparoscopic surgery22,23. A single assessment is open to subjectivity and bias, although additional criteria can improve reliability and validity. An example of this is the Objective Structured Clinical Examination (OSCE), a method of assessing the clinical skills of history taking, physical examination and patient–doctor communication68.
Martin et al.69 developed a similar approach to the assessment of operative skill, the Objective Structured Assessment of Technical Skill (OSATS). This involves six tasks on a bench format, with direct observation and assessment on a task-specific checklist, a seven-item global rating score and a pass/fail judgement. Twenty surgeons in training of varying experience performed equivalent open surgical tasks on the bench format and on live anaesthetized animals. There was excellent correlation between assessment on the bench and live models, although test–retest and inter-rater reliabilities were higher for global scores, making them a more reliable and valid measurement tool70,71. However, a global rating scale is generic and may ignore important steps of a particular operation. Eubanks et al.72 developed a procedure-specific scale for laparoscopic cholecystectomy with scores weighted for completion of tasks and occurrence of errors. For example, liver injury with bleeding scored 5, whereas common bile duct injury scored 100. Three observers rated 30 laparoscopic cholecystectomies performed by trainees and consultant surgeons. Correlation between observers for final scores was good, although correlation between final score and years of experience was only moderate.
A similar approach identified errors made by eight surgical registrars undertaking a total of 20 laparoscopic cholecystectomies73. The procedure was broken down into ten steps such as ‘dissect and expose cystic structures’ and ‘detach gallbladder from liver bed’. Errors were scored in two categories: interstep (procedural) errors involved omission or rearrangement of correctly undertaken steps, and intrastep (execution) errors involved failure to execute an individual step correctly. There was a total of 189 separate errors, of which 73 (38·6 per cent) were interstep and 116 (61·4 per cent) intrastep. However, only 9 per cent of the interstep errors required corrective action, compared with 28 per cent of intrastep errors.
All of the above rating scales are complex and time consuming; for example, the assessment of 20 surgical trainers on the OSATS required 48 examiners for 3 h each69. Furthermore, the scales are open to human error and not entirely without subjectivity. To achieve instant objective feedback of a surgeon's technical skills, virtual reality simulation may be more useful.
Virtual reality simulators as assessment devices
Studies to confirm the role of virtual reality simulators as assessment devices have concentrated on the demonstration of construct validity, with experienced surgeons completing the tasks on the MIST-VR® significantly faster, with lower error rates and greater economy of movement scores74. A direct comparison of performance is possible as all surgeons complete exactly the same task, without the effects of patient variability or disease severity. The tasks can be carried out at any time and further processing is not required to produce a test score. This can lead to the development of criterion scores that have to be achieved before operating on real patients.
At the American College of Surgeons' meeting in 2001, Gallagher et al.75 described the performance of 210 experienced laparoscopic surgeons on two trials of the MIST-VR®. The aim was to benchmark performance of these surgeons to confirm future use as an assessment tool. The results revealed marked variability in the scores obtained, together with a significant learning effect between trials. To use such data for high-stakes assessments, perhaps a pool of expert scores from all centres currently using virtual reality simulation might lead to the development of an international benchmark for trainee surgeons. Furthermore, as some trainees take longer to achieve predefined levels of proficiency than others51, this may enable particularly gifted trainees to be fast-tracked into an advanced laparoscopic programme, and the true development of a competency rather than a time-based curriculum.
Comparison of assessment tools
Currently there is no consensus regarding the optimal assessment tool for laparoscopic procedures, and perhaps video-based and dexterity systems should be used in conjunction. The authors' department has recently developed new software to enable the ICSAD trace to be viewed together with a video of the procedure, leading to a dexterity-based video analysis system76. This still requires an investment of time to assess the procedure on a rating scale, but it may be possible to identify areas of poor dexterity and to concentrate on video-based assessment of these areas alone. Other tools, such as eye-tracking devices, may also be used to highlight important areas of the operation.
Conclusion
In the American Journal of Surgery in 1991 it was stated: ‘No surgical technique in recent memory has generated as much excitement and enthusiasm among general surgeons as has interventional laparoscopy … the access may be minimal, but the operations and potential for complications are major’19. The introduction of laparoscopic techniques in the past 15 years has had a phenomenal impact on modern surgery. Laparoscopy is now the ‘gold standard’ for cholecystectomy and is making major inroads elsewhere. However, regulatory bodies have emphasized that the expanding scope of this technology should be coupled with validated training programmes incorporating inbuilt measures of performance before progressing to real procedures.
To ensure that virtual simulation can be incorporated into current training programmes there is a need to develop validated curricula for basic, intermediate and advanced level laparoscopic training. Current simulators should enable basic and some forms of intermediate level training, although the methods for achieving this remain unclear. The correct frequency of training and whether tutors need to be present at all times should be determined. Competency levels should be defined using at least a national approach, enabling standardization of training programmes. It is also important to ensure that virtual reality simulation is seen as an adjunct to traditional methods of training, and not as an alternative. Simulators currently have the ability to teach basic laparoscopic skills, enabling novice surgeons to progress along the early part of the learning curve before entering the operating theatre. With further developments in technology it may be possible to practise complete procedures, such as Nissen fundoplication and colectomy. However, surgeons will still need to reach expert levels of skill in the operating theatre; further training to achieve this is essential.
The organization of training programmes must also be addressed. It is not necessary for every hospital to have a virtual reality simulation laboratory; training can be provided at regional skills centres. Currently, training occurs in the form of isolated 2- or 3-day courses but, with the establishment of regional centres, it should prove possible to develop training programmes over periods of months and years. Junior surgeons might initially attend an intensive basic laparoscopic skills programme, followed perhaps by one session per week to learn intermediate and advanced skills. This could be aligned with their operating theatre experience, and their performance might be charted using the previously mentioned assessment tools.
A recent BJS leading article commented on surgical simulation as a ‘good idea whose time has come’77 and there is a growing confidence in its ability to revolutionize surgical education. The widespread introduction of laparoscopic procedures should be handled in a controlled and audited manner. Training programmes should exist and proctoring should remain an important part of the learning process. Only with this structured and defined approach can the complications of the last decade be avoided.
