MODEL OF SYSTEM FAILURE

A system model that helps to explain error maintains that errors are consequences rather than causes—that is, as Reason^2,3 argues, they are derived from systemic factors rather than human failings. Humans are fallible and errors are to be expected even in the best organizations.^2,3 Errors arise in “upstream” systemic factors such as policies and procedures that make error easier and safe practice more difficult. For example, intravenous infusions account for a significant portion of the adverse drug events in hospitals.⁴ When non-standard concentrations of intravenous solutions are used, mixing errors are more likely. When the floor stock of intravenous infusions is extensive, “look alike” errors increase. Humans may make the active errors that precipitate the adverse event—administration of the incorrect concentration or the wrong drug altogether—but these are symptoms of the latent errors that exist in the system. This conceptual approach concentrates on the conditions under which humans work. It seeks out and removes the error provoking properties within the system at large. This is accomplished by addressing weaknesses in the system defences against error (fig 1).

Central to the system model of error reduction is the concept of system defences. All hazardous technologies, including medical care, possess layers of barriers and safeguards. These defences take a variety of forms: technological, administrative, educational. Most important is culture—the cluster of attitudes, beliefs and values held by the individuals of the organization. Ideally, in combination the defensive layers are impermeable. In reality there are always weaknesses in the defences. The layers are more like slices of Swiss cheese containing many holes that are continually opening, shutting, and altering their positions. Fortunately, because there are multiple layers, the presence of holes in any one “slice” normally does not cause a bad outcome. If there is a hole in one, another defence will prevent the hazard from causing damage. For a mishap to occur, the holes in many layers have to line up, at least momentarily, to permit a trajectory of accident opportunity-bringing hazards. When an adverse event occurs, the important issue is not who blundered, but how and why the defences failed. We believe that a similar model can be applied to the training of a physician (fig 1).

This approach contrasts with the traditional person oriented approach dominant in health care where the focus for errors and unsafe acts is on personal responsibility of those at the site of care delivery. Personal failures such as carelessness are considered aberrant mental processes at best, and moral failings at worst. In a just world only bad people do bad things. Efforts to reduce error thus emphasize blame, disciplinary measures, retraining, and new policies, but ignore the context in which the error occurred. Reason contends that this approach has serious shortcomings, is ill suited to the medical domain, and may be counterproductive.

Medical schools have mechanisms—or “organizational defences”—to prevent the qualification/graduation of physicians who lack the necessary competencies to practice medicine. These organizational defences include entrance requirements, the curriculum, the organizational culture of training environments, student assessment, and program evaluation (table 1). We must identify and address the holes in these defences that might result in the graduation of physicians who are unable to improve care and patient safety.

ORGANIZATIONAL DEFENCES

Entrance requirements

Choosing individuals who can become the best physicians is a critical step.^5,6 Does the current process address the core abilities needed for improvement? Admission to medical school is very competitive and differentiation between individuals is difficult. A process based on grades and test scores is the primary method used because of its purported ability to predict performance in medical school. In fact, grades and scores are particularly good at predicting classroom performance (which is tested in the same way), and much less good at predicting clinical performance (which tends to be assessed differently).^7,8 Moreover, while such examinations may be useful in demonstrating some aspects of competence, they are not useful in assessing the extent to which individuals can adapt to change, generate new knowledge, and continue to improve their performance.⁹

The quantitative approach to candidate selection is convenient as it allows one to rank applicants and schools (box 1) using standardized numbers. This is supplemented by qualitative data collected from student essays, letters of recommendation, and interviews meant to inform the admissions committees about the integrity, values, commitment, communication skills, and interpersonal skills of the applicants. Such qualitative data have their own limitations and quantitative data easily become dominant. This may have pernicious effects such as competitiveness and grade consciousness among students applying to medical school. For example, a study of students at Harvard Medical School¹⁰ found that the so called “premedical” syndrome—in which premedical students become study machines and are characterized as hypercompetitive, narrow minded, greedy, and dishonest at best and “ferocious geeks” at worst—was in evidence.

Although considerable effort is made to ensure that medical students have the potential to become good physicians, up to now this has not explicitly included the attitudes and behaviors needed for improving care and safety. If students lack the integrity and willingness to engage in honest self-reflection, a school of medicine must try to instill those values. If students lack the ability to work with others, it is left to the medical school to help them to learn how. These are daunting tasks. A better organizational defence might be to include an assessment of prior experience and skills in these areas as part of the process of student selection.

Curriculum

The content of a typical medical school curriculum includes the sciences basic to medicine, a variety of clinical disciplines, and ethical, behavioral, and social subjects pertinent to medicine. This extensive curriculum equips qualified doctors to care for individual patients, one at a time. Less attention is paid to the skills necessary to improve one's practice as a whole.^1,11 Table 2 summarizes the core domains of knowledge needed for competency in the improvement of health care.

Improvement in healthcare delivery (even one's own practice) is almost always an interdisciplinary process, requiring the expertise and collaboration of everyone who works in the system to be redesigned. The professionals involved must be ready to contribute their own knowledge and skills and be willing to learn from the expertise of others. Each must also be aware of differences such as language and methods of work that, unrecognized, can interfere with effective collaboration.

High reliability industries such as aviation and nuclear power recognize this need and their training emphasizes teamwork.^12,13 For example, in aviation “crew resource management” (CRM), originally termed “cockpit resource management”, emphasizes such training. A study by Sexton et al,¹⁴ which compared the views about teamwork of flight crews and hospital operating room personnel, is telling. In this study, which involved surveys performed over several years in several countries, aircrew and operating room personnel differed substantially in their view of how teams should work. While there was overwhelming support for flat hierarchies by airline crew members (94%), only 55% of attending surgeons preferred flat hierarchies. Moreover, different operating room personnel perceived the level of teamwork very differently. Surgical residents and attending physicians reported high levels of teamwork (73% and 64%, respectively), but only 39% of attending anesthesiologists, 28% of surgical nurses, 25% of anesthesia nurses, and 10% of anesthesia residents reported high levels of teamwork. High reliability industries also recognize that human flexibility to adapt to changing events is an important safeguard against catastrophic accidents. Their training emphasizes the rehearsal (simulation) of both familiar and novel scenarios of failure. It is through this type of training that the ability to perceive and work effectively in interdependencies can be fostered.

Examples of team training and simulation in health care are relatively sparse. Training in advanced cardiac life support includes some of these elements and is one of the few instances involving large numbers of individuals. There are some examples of applying the CRM approach, primarily in the training of anesthesiologists and in the intensive care unit, emergency room, and labor and delivery room settings.^15–18 Notwithstanding the limited number of studies in health care, the successful implementation of CRM in aviation suggests that this method has something to offer the training of physicians.

Systematic implementation of interdisciplinary training of any kind has not taken place in medical education,^19,20 despite many examples of interdisciplinary efforts that successfully improved care.^21,22 If interdisciplinary training is to be implemented, we suggest that it should begin early, before learners become “isolated in their discipline specific domains and `tainted' by traditional disciplinary hierarchies, boundaries and biases”.²³

Medical school curricula lack another important topic that constitutes a critical part of the way that high reliability organizations prevent error. In high reliability industries, near misses that result from error are considered an opportunity to improve the system as a whole.¹³ Although most physicians would state that they learn from their mistakes, teaching about the value of error is not included in most medical curricula.²⁴ Moving away from an approach of “name, blame and shame” towards a “no blame” scenario where errors can be more openly admitted and discussed can help to develop an atmosphere where we all can learn from error. In addition, it can foster a sense of responsibility for error and responsibility for reporting it. However, it is important to recognize the emotional costs of involvement with a medical error and the need for appropriate emotional support. Where do medical students learn to perceive error as a character flaw? Apart from what individuals bring with them when they enter medical school, they are socialized to the profession and to the cultures of the organizations in which they train.

Organizational culture

Organizational culture strongly influences practice,²⁵ including the ability to improve care and patient safety. We consider only two of the cultures in which medical students are socialized—the culture of the medical profession and the culture of the medical school and academic medical center. Socialization refers to the processes by which individuals come to understand and internalize the attitudes and values inherent in a particular social role and which are distinct from those of society in general.²⁶ This helps to explain the behavior of individuals in work settings.

The high financial, social, economic, and psychological investment involved in the arduous training of a medical student promotes strong bonds with those who have the same interests and experiences. Sinclair found that physician/physician cooperation was paramount throughout the years of training.²⁷ Tribalism is encouraged by the apprenticeship style of medical education in which students learn to be doctors as part of tightly knit physician teams, especially in the hospital. This important learnt disposition may help to explain the reluctance of physicians to criticize other physicians and why, when challenged, they turn to each other rather than seek help outside the medical network.^24,27 Doctors may also turn to each other when faced with a medical error because of the learnt disposition of the value of experience (gained by seeing patients) and responsibility (gained by having patients and treating them). Feelings of elitism, collegiality, and exclusivity of professional judgement foster a belief in the necessity of self-regulation: no one else can understand. The result may be professional leniency, contributing to future medical errors.

Professionalism in medicine, defined by the American Board of Internal Medicine (ABIM), requires the physician to serve the interests of the patient above his or her self-interest.^28–31 Professionalism aspires to altruism, accountability, excellence, duty, service, honor, integrity, and respect for others. This is the ideal that is espoused. Learning about the reality occurs in what has been termed the “hidden” or informal curriculum.³² In fact, the socialization into medicine may result in a cynical approach of “us versus them” (see, for example, House of God by Shem). The ABIM's Project Professionalism cited a variety of challenges that threaten professionalism, such as those relating to chronic fatigue and sleep deprivation; stress and overwork; lack of confidence, self-esteem, and experience; difficult patients; chaotic, unstructured, unsupervised rotations; creative tension with other health professionals and lack of professionalism among house staff; arrogant faculty; health risks to the profession; abuse of power, and family obligations.²⁸ Unless the medical education system deals effectively with these challenges, there may be holes that allow the graduation of individuals without the values needed to improve care and prevent error.

Medical schools and academic health centers are among the most important contributors to the culture of the practice of medicine. Situating medical schools in a university framework ensures that learning will take place in a research environment.³³ This exposes students to the spirit of inquiry and its requirement for critical analysis, both of which are essential to the scientific practice of medicine. In learning to be a physician, students try to learn the ways of science as well as mastering a specific body of knowledge. However, teaching the skills required to improve care and patient safety requires qualified teachers. A “hole” in this concept is the current reward system for faculty, with its emphasis on research. In many medical schools, research activities are valued far more highly than improving one's own clinical practice or teaching.³⁴ Academic careers depend primarily upon research publications and grants. Yet there is far more involved in the education of a medical student than ensuring that such education takes place in an environment of research and discovery. James J Duderstadt, President Emeritus of the University of Michigan, stated: “At the root of our unmet challenge in undergraduate education is the failure to distinguish between the transmission of knowledge and the development of a capacity for inquiry, discovery, and continued learning. The predicament is that the faculty is transmitting what they know and love with little awareness of what the student needs to learn”.³⁵ The issue in medical schools is the same. It has been said that “you are what you practice most”. If an institution values research above all, one should not be surprised that this value is transmitted to everyone in the institution, including the students training there.

Medical students learn how to deal with error from their teachers—physicians who are themselves training in graduate medical education programs plus attending or staff physicians—as well as from the culture of the organization(s) in which they train. A study of physician trainees found that they used three major mechanisms to define and defend medical error: denial, discounting, and distancing.³⁶ Similar observations have been made by others.^37,38 As already noted, what underlies these mechanisms is the assumption of the personal model for error; all that can be done is to be more careful next time. A marked cultural change will be necessary to create the kind of safety culture that characterizes high reliability industries. In such cultures individuals still take personal responsibility, but it doesn't end there. Safety cultures acknowledge that error will occur, value near misses, learn from error, and have respect for expertise regardless of rank. They are marked by “interpersonal responsibility; person centeredness; (co-workers) helpful and supportive of one another; friendly, open sensitive personal relations; creativity; achieving goals, strong feelings of credibility; strong feelings of interpersonal trust; and resiliency”.³⁹ Similarly, a culture of improvement supports measurement and reflection in practice, searches for the gap between what is and what could be, and rewards those who work to close that gap.

It is clear that medical students learn within several cultures, some of which may be easier to change than others.²⁶ The underlying assumptions (such as the longstanding hierarchy of the professions) and values (such as those that inform the hidden curriculum) are deep seated. To have physicians finish their training prepared to improve care and patient safety, medical schools may need to favor training opportunities in healthcare organizations actively building the required environment and culture.

Evaluation of learners and programs

Student assessment

Medical student assessment seeks to provide a comprehensive evaluation that ensures the graduation only of those prepared for the practice of medicine. A variety of methods are used, but few address the competencies related to improving care and patient safety. Murray et al⁸ point out that there are extensive tools for assessing clinical skills and knowledge, but fewer (and less well developed) techniques for assessing professional behavior, scholarship, and multiprofessional teamwork. There is a mismatch between what we measure and our real objectives. Furthermore, one might question the correlation between scores on “objective” (particularly multiple choice) examinations and actual performance in practice.

One major advance is the Objective Clinical Examination (OSCE).^8,40 The OSCE is a valid and reliable method of assessing a learner's ability to apply clinical knowledge and skills in simulations of clinical encounters. Examinee performance is directly observed, albeit in a simulation. Alternatively, one might use actual patient outcomes to assess trainee performance. This also has its limitations. These outcomes depend in part on physician action, but are also affected by patient and system factors. For medical students the link between student action and patient outcomes is even more distant.

To help bridge these gaps, Coles⁷ suggests that we develop (rather than ignore or dismiss) the subjective judgments we make as we observe trainees in practice. This is a promising approach, but we have more to learn for such qualitative ratings to be sufficiently reliable and valid.

It is important to remember the maxim: “Tell me how I am being measured and I will tell you how I will behave”. Assessment drives learning; students will prepare for what they think will be on the examination. Holes in the defences arise when there is a mismatch between what we measure and what our real objectives are. Exclusive reliance on multiple choice examinations like the National Board of Medical Examiners (NMBE) allows the graduation of students who excel on standardized tests but may lack other key skills.⁹ The Accrediting Council on Graduate Medical Education (ACGME), which accredits programs in graduate medical education in the US, has adopted a broader competency based approach. They now require programs training physicians in all specialties to assure competency in six core areas, including practice based learning and improvement.⁴³ Developing valid and reliable measurements for these assessments is an area of active work. This more comprehensive approach will increase the chance that physicians will finish their training prepared to improve patient care and safety.

Program accreditation

The Liaison Committee on Medical Education (LCME) accredits programs of medical education leading to the MD degree in the US and Canada. Its process is designed to assure “that medical schools meet current national standards of structure, function, and performance, so that students receive a valid educational experience and governmental agencies have a reliable criterion for licensure”.⁴² Programs must meet standards for accreditation “so that its graduates will be prepared to enter and complete graduate medical education, to qualify for licensure, to provide competent medical care, and to have the educational background necessary for continued learning”. Similar processes operate in other countries. However, most accreditation standards are, to a large extent, still tied to the traditional model of training doctors to care for individual patients. Medical schools are not held accountable for the full range of physician competencies needed. Educators must be asked to show that graduates possess the skills and attitudes needed to improve care. As with the assessment of individuals, an expanded assessment of programs requires a major effort to develop the appropriate tools and methods.

CONCLUSIONS

The graduation of physicians prepared to improve care and patient safety is no accident. It requires a unified system of education (entrance requirements, the curriculum, the culture of training environments, student assessment, and program evaluation) to ensure the development of the knowledge, attitudes, and skills needed. An analytical approach borrowed from the field of human error allows us to examine this system of education critically. In doing so, we can identify and address “holes in the defences” and increase the number of new physicians prepared to reflect on and continually improve professional practice.