Article Text

Download PDFPDF

Beyond “see one, do one, teach one”: toward a different training paradigm
  1. J M Rodriguez-Paz1,2,
  2. M Kennedy3,
  3. E Salas4,
  4. A W Wu2,5,
  5. J B Sexton1,2,5,
  6. E A Hunt1,2,6,
  7. P J Pronovost1,2,5
  1. 1
    Johns Hopkins University School of Medicine, Department of Anesthesiology and Critical Care Medicine, Baltimore, MD, USA
  2. 2
    Johns Hopkins University School of Medicine, Quality and Safety Research Group, Baltimore, MD, USA
  3. 3
    Johns Hopkins University School of Medicine, Department of Surgery, Baltimore, MD, USA
  4. 4
    University of Central Florida, Department of Psychology & Institute for Simulation & Training, Orlando, FL, USA
  5. 5
    Johns Hopkins Bloomberg School of Public Heath, Department of Health Policy & Management, Baltimore, MD, USA
  6. 6
    Johns Hopkins Simulation Center, Baltimore, MD, USA
  1. Dr J M Rodriguez-Paz, Department of Anesthesiology and Critical Care Medicine, 600 North Wolfe St/Meyer 297A, Baltimore, MD 21287-7294, USA; jrodrig1{at}jhmi.edu

Abstract

In the process of acquiring new skills, physicians-in-training may expose patients to harm because they lack the required experience, knowledge and technical skills. Yet, most teaching hospitals use inexperienced residents to care for high-acuity patients in complex and dynamic environments and provide limited supervision from experienced clinicians. Multiple efforts in the last few years have started to address the problem of patient safety. Examples include voluntary incident-reporting systems and team training workshops for practising clinicians. Fewer efforts have addressed the deficits in training new physicians, especially related to knowledge, skills and competence. The current apprenticeship or “see one, do one, teach one” model is insufficient because trainees learn by practising on real patients, which is particularly an issue when performing procedures. Residents have expressed that they do not feel adequately trained to perform procedures safely by themselves. In this paper, we conduct an informal review of the impact of current training methods on patient safety. In addition, we propose a new training paradigm that integrates competency-based knowledge and clinical skills, with deliberate attitudinal and behavioural changes focused on patient safety in a safe medically simulated environment. We do so with the hope of creating a better marriage between the missions of training and patient safety.

View Full Text

Statistics from Altmetric.com

The intern suffers not only from inexperience, but also from overexperience. He has in his short term of service responsibilities which are too great for him; he becomes accustomed to act without preparation and he acquires a confidence in himself and a self-complacency which may be useful in times of emergency, but which tends to blind him to his inadequacy and to warp his career. (William Stewart Halsted, 1904).

Quality of care and patient safety have become a significant focus of medical practice. This focus was accelerated after the Institute of Medicine’s report in 19991 described medical errors as the eighth leading cause of death in the US and recommended the development of newer and safer ways to deliver care to patients.

Physicians-in-training (PIT) may expose patients to harm because of lack of experience, knowledge, and technical skills while working in a complex and dynamic environment with limited resources and supervision.2 To advocate for a more robust training model than our current model of “see one, do one, teach one,” we conducted an informal literature review of the impact of current training methods on patient safety. The search terms entered into PubMed included: patient safety, medical errors, patient simulation, medical education, clinical competence and procedural errors.

In addition, we propose a new model for ensuring competency and safety of PIT, which is based on the science of patient safety and simulated training. However, the feasibility, efficiency and effectiveness of this new model require further evaluation.

EVIDENCE FOR SHORTCOMINGS OF THE CURRENT TRAINING SYSTEM

Most teaching hospitals use PIT to care for high-acuity patients in the most complex clinical areas, such as the operating room, emergency department and intensive care unit (ICU). In all three clinical areas, patients are physiologically vulnerable and less likely to recover from mistakes.38

Some of the deficiencies of the current medical training practices are listed in box 1. Furthermore, besides these deficiencies in training and education, other factors which may lead to medical mistakes include failed or ineffective communication and teamwork, inadequate supervision, and failure to follow or lack of established protocols.911 Baldwin et al12 found that a significant proportion of mistakes made by residents were attributed to insufficient knowledge or inexperience. The Joint Commission for the Accreditation of Healthcare Organizations (JCAHO) cited inadequate training/orientation as the root cause in over 50% of sentinel events and 60% of medication errors. (http://www.jointcommission.org/SentinelEvents/Statistics). These findings are consistent with results from medical simulation,111315 in which inadequate knowledge, experience, and/or supervision were reported as causes of mistakes.

Box 1 Problems of current medical training practice

  1. High-risk and stress environments

  2. Lack of practice on a regular basis, especially with uncommon events/procedures and/or complications

  3. Lack of rigorous evaluation, readjustment and correction of problems

  4. Limited knowledge of the impact that medical training has on patient safety

  5. Unlike other high-reliability industries, we train on real patients and must “assume” the inherent risks

  6. Lack of training in team interactions, crisis management, and conflict resolution

  7. Perpetuation of the “see one, do one, teach one” method

Studies show that between 28% and 42% of residents felt inadequately trained to safely perform a medical procedure alone for the first time.1617 Wu et al18 found that 45% of internal medicine house officers reported making at least one mistake that, in 31% of the cases, were fatal. Only half of these were reported to the attending physician, and a quarter were reported to the patient or family. In another study, 29% of internal medicine residents felt inadequately trained to care for mechanically ventilated patients,19 and 44% still expressed limited knowledge in managing these patients at the end of their residencies.

One particularly common type of error among PIT involves therapeutic and diagnostic procedures. We know relatively little about the epidemiology of procedural errors, especially because errors are under-reported1 and because most of the available data were obtained from self reports, which significantly underestimate the magnitude of errors.20 In the Medical Practice Study, procedural and therapeutic mishaps were the second most frequent cause of adverse events.821 Another recent study in two Boston academic hospitals2 found that the most common medical errors reported by residents involved procedures (31%), with 25% being fatal or life-threatening. In addition, inadequate supervision was noted in 20% of the reported cases. These findings are similar to those in Wu’s study,22 which described resident inexperience as an important contributor in 11% of cases involving procedures. In this study, the authors also found that even when residents make mistakes, only half report them to their attending, and only a quarter tell the patient’s family. A culture in which PIT openly identify and learn from mistakes may move us in the right direction.

The Critical Care Safety Study Group recently reported that 80% of performance failures (ie, adverse events and serious medical errors) were attributed to skill-based slips and knowledge-based mistakes.23 Moreover, based on error rates found in teaching hospitals, this group estimated that 148 000 life-threatening, serious errors occur annually in the United States. Recent survey results from a statewide ICU quality improvement collaborative24 showed that training, supervision and patient harm were significant issues of concern. Only one of three respondents (residents and fellows) strongly agreed that they were adequately supervised, and that their hospital does a good job of training new staff. Further, 31% of these respondents reported making mistakes that had the potential to harm patients. These results suggest that PIT perceive current methods for training physicians to be inadequate, and given the high rates of errors, patients would likely agree.

Information contained in patient safety reporting systems is also alerting us to problems with procedure-related errors. Procedure errors were common in the web-based ICU safety reporting system (ICUSRS). The ICUSRS was a prospective cohort study of incidents reported by nurses and physicians.2527 Of 2075 incidents reported from 23 ICUs in the ICUSRS study, 1017 (49%) noted deficiencies related to training and education.25 Furthermore, residents were involved in 198 incidents (9.6%), of which 169 (87.5%) were considered preventable. In 77 (3.8%) of the incidents, deficiencies in training, knowledge and/or skills were identified as contributing to the error. Within the subset of training/education incidents, the degree of patient harm varied from moderate (15%) to severe (36%) physiological changes. In 16.4% of incidents, the error caused physical injury, and in one instance, the patient died.

In summary, data from empiric studies, surveys, and incident reports suggest that residents are inadequately trained to perform procedures; inexperienced and insufficiently supervised residents harm patients.

CURRENT MEDICAL TRAINING PARADIGM: SEE ONE, DO ONE, TEACH ONE

The current model for medical training mimics the classic apprenticeship model of “learning by doing,” commonly known as the “see one, do one, teach one” model. The principle behind this model is that experience facilitates learning while simultaneously promoting trainee autonomy and education.

This traditional paradigm, however, lacks important components to ensure that trainees are competent to practise medicine, including consistent guidance, measurement of performance and feedback in a systematic and structured way. In fact, the current model assumes that a trainee is competent after performing a specific number of procedures or completing a predetermined number of postgraduate years. In many countries, including the US, trainees rarely undergo formal assessment of competencies and/or proficiency with procedural skills, and in many cases, assessment of the competencies takes place after residency; therefore, doctors can practise independently for a number of years without passing any formal examination. The concept of competence-based advancement rather than time served must become the standard of medical training and be incorporated into current certification processes.28 Moreover, trainees may have limited exposure to skills such as teamwork, communication, leadership and management, especially in acute environments like the ICU.29

A variety of methods have been introduced or recommended to improve current training programmes. These include the Problem-Based Learning model, inclusion of a competency-based training model by the Accreditation Council for Graduate Medical Education (ACGME, http://www.acgme.org) and the “Competency-Based Training Programme in Intensive Care” (CoBaTrICE) model (http://www.esicm.org/PAGE_cobatrice?13ft). However, these new training methods may still fall short of their intended goal because their impact on patient safety has not been rigorously evaluated. In addition, few of these models incorporate a framework for safety culture30 that can be assessed with widely used validated tools and linked to clinical and operational outcomes, like the Safety Attitudes Questionnaire (SAQ).3132 As the science of safety matures, so too should our use of this knowledge in preparing residents for the growing complexities that they will encounter as clinicians. Recently, other factors that may play a role in PIT performance and medical errors have been analysed,33 including fatigue.183436 To decrease fatigue and improve patient safety, the ACGME implemented work-hour restrictions. In a paper by Landrigan et al, the authors found that while implementation of these restrictions showed a decrease in the number of serious medical errors in the ICU,37 the frequency of errors made by “rested” house officers remained unacceptably high (158.4 per 1000 patient-days) even for procedure-related errors. Moreover, in a recent article, Barger et al,38 found that although the number of errors and adverse events were higher among those interns who were working extended shifts, even those interns who followed the 80-hour rule had a significant number of self-reported medical errors. Undoubtedly, the number of errors increases with longer work shifts and fatigue, but these studies suggest that lack of knowledge, skills, competence and/or supervision—not only fatigue—played an important role in causing errors by the PIT.

BEYOND “SEE ONE, DO ONE, TEACH ONE”: TOWARD A NEW TRAINING PARADIGM

We propose that a training model must integrate knowledge and skills-based learning with a culture of patient safety in both simulated and “real” environments. It should integrate four principles: understanding (knows), application (knows how), integration (shows how) and practice (does).39 The model must assess knowledge and skills and test attitudes about safety. This model should migrate from an “action–reaction” to an “action–reflection” process that more realistically mimics the processes by which humans learn.4041

Though several competencies could be included in this new model, it would be unrealistic to develop a curriculum that included all of the theoretical competencies that define the practice of PIT. Instead, we propose that defects or mistakes should inform curriculum development. For example, we could identify defects, that is, inadequately performed tasks that pose risks to patients, through incident reports, morbidity and mortality conferences, liability claims or investigative safety tools (ie, the “Learning from Defects” tool).42 This training paradigm would involve learning, repeated practice, performance and evaluation (fig 1). An example of how to apply this paradigm is illustrated in fig 2. This model could be implemented to remediate a typical defect, that is: complications associated with the removal of central lines.43

Figure 1 Components of a new training model.
Figure 2 New training paradigm: the case of central line placement training.

In this model, residents would first complete a goal-oriented, knowledge-based curriculum that achieves competency as defined by the ACGME (http://www.acgme.org/outcome/comp/compMin.asp). After completing didactic curriculum, residents would practice what they learnt using medical simulators. Medical simulation allows PIT to repeatedly “practice on plastic first” and play out a wide variety of scenarios and error-prone situations and to reflect on performance without jeopardising a patient’s safety, while providing a controlled setting in which rigorous skill assessment and feedback occur to help trainees to develop clinical competence.44 Trainee performance will be measured (Practice/Evaluate/Correct/Feedback) and skills re-practised to gain proficiency.

Once the resident is proficient in the defined learning outcome(s), they are ready to perform the practised skill on real patients under close supervision by senior residents or attendings. The supervisory role could be filled by trained simulation clinical specialists (including physicians, nurses, fellows, etc) One practical strategy for this process is the use of briefings and debriefings. Peers can discuss plans prior to a procedure and reflect on their performance during a debriefing after it. Any defect in the trainee’s performance will be corrected, and if necessary, the PIT can return to the simulator to practise. Once proficient, trainees can close the loop of training by instructing or supervising other trainees.

As with any programme, this training model must be evaluated. Learning outcomes, changes in attitudes (safety culture) and real-life outcomes can be measured using the four levels of Kirkpatrick’s model,45 which is consider by most as the standard for evaluating the efficacy of training. First, measurement of how residents react to the training can be accomplished via satisfaction and programme evaluation surveys.4648 These surveys will measure quality of simulated scenarios and perceived utility in real life.

Second, what residents have learnt from the simulation must be measured and can be accomplished by using pre- and post-testing, observation by simulator- and non-simulator-trained trainers, through debriefing tools. Standard performance metrics borrowed from other disciplines can be used to evaluate technical and team skills. Though the development of performance measures is immature and requires resources, many specialty boards and professional societies are making progress. Performance metrics have been validated in different medical specialties, including anesthesiology, surgery, obstetrics, emergency medicine, critical care medicine, paediatric resuscitation, dermatology, bronchoscopy/colonoscopy, etc.4962 It is important to note that different skills for different competencies will require different measures. A detailed description of these measurements is beyond the scope of this paper, but some of these measurements include pretest–post-test design, global rating scales, team-based “360-degree” assessment, checklists, Objective Structured Assessment of Technical Skill (OSATS), etc.(ACGME Outcomes Project Toolbox of Assessment Methods, http://www.acgme.org/Outcome/assess/Toolbox.pdf).6366 Given the complexity of many procedures performed by PIT, the number of potential performance measures could be overwhelming. However, by focusing on the steps in the process that go awry and result in preventable harm, educators can prioritise where to measure and focus on processes that could improve patient safety.

Third, it must be assessed whether the training acquired transfers to everyday practice by measuring whether the skills learnt are used and how practice changes. This is done by evaluating PIT and the teams in which they work, using behavioural markers.5067 As part of this assessment, the impact of this new method could be measured on patient outcomes, on observed behaviours, (ie, process measures) or by measuring reductions in errors/defects in sentinel events related to the trained competency.68 It is important to recognise that information obtained from patient safety reporting systems (PSRS) helps to identify hazards; they should not be used to evaluate progress in patient safety, as the rates of events in PSRS are likely biased by self-reporting and poorly defined populations at risk.69

LIMITATIONS

The model proposed in this paper has several potential limitations, the most obvious relates to issues of cost and implementation. This model of training requires resources, including trainers, simulators, time, space, implementation of new technologies and the creation of specific curriculum for multiple procedures/competencies for various medical specialties. These resources are costly. Although data are lacking on the cost-effectiveness of simulation-based training, in this age of lawsuits and raising awareness of adverse events and medical errors “the best safe practice is good for business.”70 Given that we currently have insufficient numbers of experienced clinicians who can provide the time required to supervise and train PIT, this model may seem unrealistic. The training provided during simulation would likely be provided not by clinicians who currently supervise PIT but by educators and trained simulation specialists. By using simulation, PIT may arrive on their clinical rotations with more experience and require less supervision. To be certain, this model will require resources. Yet, when the societal costs of having inadequately trained physicians is considered, the return on society’s investment in training is likely to be substantial.

It is clear that the effectiveness of this model relies on a change in culture that affects both individuals and institutions. Individual and institutional effort will be required to move from the traditional methods of training to a more innovative paradigm that is focused on patient safety and quality improvement. In addition, institutional effort and resources will be required to integrate simulation training into training programmes. A significant component of the culture change is for clinicians to view their work as a process. The use of this model will require clinicians to articulate the steps required to perform a procedure (ie, the steps in the process), identify where those steps break down and develop training to defend against those mistakes. Paradigm shifts require a level of understanding of how processes work, which will present a challenge for those who are “process illiterate.”

Another major limitation of this training paradigm is the lack of fully developed assessment tools. The science of developing process tools is immature, though many are working on them. Because the resources required to develop valid yet feasible assessment tools are significant and likely exceed those available at most hospitals, specialty and professional societies can play an important role in developing them. Valid and reliable evaluation tools are required to assess the technical, leadership and teamwork components that contribute significantly to adverse events.313371 Such tools are beginning to emerge from the societies for professionals in anaesthesia, surgery, emergency medicine and obstetrics, among others. Other valid assessment tools include the well-established Objective Structured Clinical Examinations and Standardized Patients assessment tools.72 In addition, if simulator training is insufficiently resourced, it could introduce or encourage incorrect practice or practices that do not translate to skills with real patients.

Lastly, good evidence is lacking concerning the effect of simulation-based training. However, despite this lack of solid evidence, multiple institutions and organisations have been pushing for this type of training. Since the publication of the 2001 Institute of Medicine report, “Crossing the Quality Chasm: A New Health System for the 21st Century”73 which recommended changing medical training by using simulation, many organisations have embodied these recommendations. The American Board of Internal Medicine, the American Society of Anesthesiology, and the American College of Surgeons are encouraging the use of simulation-based education as part of the requirement for training of their trainees. The early use of simulators was often viewed as an add-on to training rather than an integral component of ensuring competency. In our model, simulation is an essential component in ensuring the competency of PIT.

In conclusion, current medical training, especially for procedural skills, is insufficient and presents a source of preventable harm to patients. We propose that any training paradigm must integrate knowledge, skill-based competencies and a culture of patient safety, and must include a medically simulated environment. With this model, trainees learn that errors are an integral part of medical training and practice new skills in a safe, simulated medical environment. Thus, trainees are able to act and reflect on their experience before practising on real patients. This model may improve medical training and patient safety simultaneously. However, it will require resources. Given the societal costs of having inadequately trained physicians who practise for several decades, the return on investment in ensuring physician competency should be substantial.

REFERENCES

View Abstract

Footnotes

  • Competing interests: None.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles

  • Quality lines
    David P Stevens