METHODS OF CLASSIFYING ERRORS OF DIAGNOSIS

A review of the literature failed to uncover any systematic means of classifying the cause of diagnostic errors. Errors in diagnosis have traditionally been graded as major errors or class I errors, depending on whether they are a principal or underlying cause of death or whether they would have affected prognosis if detected during life, respectively.¹⁸ These grading systems give no insight into the cognitive or technical processes that lead to errors. The literature review revealed several systems that could be adapted with little or no modification for the purpose of causal classification.

The first classification system is used without modification from the Adverse Medical Event System for transfusion medicine.¹⁹ This method of error analysis is a prototype national event reporting system for blood centers and hospital transfusion services and is thought to have widespread error analysis application in industry and in medicine.²⁰ The development and application of this system for transfusion medicine has been previously described.^19,20 Briefly, the classification of root causes of medical error are divided into latent errors and active errors (table 1). Latent errors are subdivided into technical and organizational. Active errors are categorized as external and behavioral, which is further subdivided into knowledge based, skill based, rule based, and other. While this system has enjoyed success in identifying root cause of error in the field of blood banking, it has not been tested for its applicability to errors in diagnosing disease.

The second classification system is based on the “dimensions” of professional competence as outlined by Epstein and Hundert (table 2).²¹ This classification system is an inventory of professional skills and behavior considered essential for the competent practice of medicine.²¹ Given that this inventory of cognitive, technical, and higher level integrative skills characterized the dimensions of competent medical practice, it could also be used to identify the fundamental deficiencies in physicians whose performance is deemed suboptimal. The dimension of professional competence as proposed by Epstein and Hundert was modified for use in this study by removing three categories (referred to as context, relationship, and affective/moral) because of the difficulty of measuring these behavioral characteristics. The 22 subdivisions in the remaining four categories (cognitive skills, technical skills, integrative skills, and habits of mind) were condensed into 16 to reduce conceptual overlap.

The third system is borrowed from a paper by Kassirer and Kopelman¹³ who compiled a comprehensive list of faulty cognitive practices that lead to error in diagnosis (table 3). The purpose of their compilation was to identify and classify cognitive errors so that a deeper understanding of the epidemiology and causes of diagnostic error could be obtained.¹³ This so called catalogue of faulty cognition was itself derived from the scientific literature and is based on the assumption that clinical problem solving and the cognitive process of scientific discovery are inherently similar.¹³ The five categories of cognitive diagnostic errors along with the definitions proposed by Kassier and Kopelman were used without modification.

Although the origin of these three classification systems differs, each appears to have potential value in studying the cause of diagnostic errors.

STUDY DESIGN

The clinical records of five patients with ocular or orbital tumors who had less than desirable clinical outcomes were reviewed independently by three ophthalmologists who were fellowship trained in ocular oncology (see box 1 for clinical summaries) Each participant devotes at least 75% of his practice to ocular oncology. All aspects of clinical care, technical and environmental backgrounds were available. Each participant was asked if a misdiagnosis had occurred. If yes, was it minor, moderate or serious? They were asked to identify one primary cause of the misdiagnosis using three different classification systems. The participants were given several weeks to study the written case material and each classification system. They were encouraged to ask questions concerning the classification systems. Root cause analysis involves investigation into all aspects of care potentially related to the error. Because much of this material is not contained within the medical record, participants were provided access to the technical, organizational, and environmental details of each case through an interview. The participants were able to request any potentially relevant information including—but not limited to—the training and professional experience of physicians, qualifications of technical support staff, quality and condition of medical instrumentation, and type of clinical practice (private practice group, private practice solo, academic medical center, general or referral practice, etc).

The definition of misdiagnosis used by the participants in the study was a “conclusion about an abnormal state of health that leads to a management decision resulting in an otherwise avoidable injury”. The distinction between degrees of error (minor, moderate, and serious) is subjective and was left to the discretion of the reviewer.

Agreement among participants for each system was calculated by dividing the number of agreed responses by the total number of responses; proportions were expressed as percentages. A high level of agreement between participants for a given system would indicate that errors of diagnosis can be reliably classified according to primary or root cause.

The study was conducted between February and July 2002.

RESULTS

The three participants agreed that a misdiagnosis had occurred in all five cases (100% agreement). The errors were judged as serious in 14 of 15 occasions (93%) and once as moderate (7%). One participant graded the error in case 3 as moderate because the delay in making the correct diagnosis should not have changed the prognosis.

Three participants agreed on the primary cause of error twice—once in case 3 using the Professional Skills and Behavior Inventory and once in case 5 using the Medical Event Reporting System (table 4). Two of the three participants agreed on the primary cause of error three times—twice using the Medical Event Reporting System and once using the Professional Skills and Behavior Inventory (table 4). Forty seven percent of responses were in agreement using the Medical Event Reporting System and 33% using the Professional Skills and Behavior Inventory. There were no primary agreements with the Catalogue of Cognitive Error.

DISCUSSION

There was near uniformity of agreement (93%) that the errors of diagnosis in these five cases were serious. These cases are not rare events. Current analysis of autopsy data indicates that a major diagnosis goes undetected in more than 74 000 individuals who die in hospitals in the United States each year.¹⁸ Nearly 35 000 of these patients could have survived hospital discharge if the error had not occurred.¹⁸ Why do diagnostic errors occur and can such errors be prevented in the future?

There have been few formal investigations or discussions into the causes of misdiagnosis despite its importance to patient safety.^7,8,13,22 The three clinical experts in this study agreed on the primary cause of error, occasionally using three different systems for classification. Two of the systems in this study (Adverse Medical Event System and Professional Skills and Behavior Inventory) showed some potential usefulness but, even then, all three experts could agree only once with each system (table 4). Two of the three experts agreed in two other cases using the Medical Event Reporting System and in one case using the Professional Skills and Behavior Inventory. The diversity of opinion offered by these three experts may even be more noteworthy than their modest level of agreement (table 4). There were no agreements on any case using the Catalogue of Cognitive Errors even though it had only five choices.

The rather disappointing results of this study raise the question of why such a wide variety of primary causes were ascribed to each case by clinical experts. There is no simple explanation. Perhaps the categorical choices were too abstract or too conceptually overlapping. Maybe the understanding of medical decision making itself is too rudimentary. On the other hand, the multidimensional nature of decision making may not be easily categorized. Studies of complex work environments have shown that, once a minor mistake enters a system, secondary errors can propagate the problem and increase the likelihood of serious failure, particularly when no safeguards are built into the system to limit damage.⁷ Secondary errors—because of their immediate clinical consequences—can often eclipse primary mistakes in importance to the clinical outcome. The traditional domains used to categorize cause of injury such as human (behavioral), organizational, and technical also have considerable overlap in clinical practice which makes categorical distinctions potentially ambiguous.

There have been few analytical studies of diagnostic errors. Battles and Shea²⁰ performed an analysis of the root cause of medical error in a teaching hospital using the Eindhoven Classification Model (similar to the Adverse Medical Event System employed in this study).¹⁹ They found that root cause analysis can be a valuable source of information for guiding educational and system change in the hospital. Their study, unlike this one, dealt with diverse types of medical error and did not focus on mistakes due to diagnosis.

The definition of misdiagnosis in this study is based on a medical injury model and not on a medical error model.^22,23 Error and harm are not always linked. The medical error model, which includes mistakes that do not harm patients, would create too broad a definition for the purposes of studying misdiagnosis. The normal process of making a clinical diagnosis involves hypothesis testing. So called working or provisional diagnoses are often corrected as a clinical work up proceeds through logical stages. While there is considerable controversy over the roles served by the medical injury and medical error models in improving patient safety,^22,23 the medical injury model has the practical advantage of establishing a less controversial threshold for the study of misdiagnosis by circumventing medical errors that cause no harm. Because of physician sensitivity and concerns over medicolegal liability, pilot studies of diagnostic error would be better received if based on the medical injury model.

The rationale for this study is based on the assumption that the opportunity to detect preventable morbidity and mortality increases when medical errors are defined in terms of failed processes rather than personal deficiencies. The results of this study indicate that, in an idealized situation (in which concerns over personal and professional reputation and medical liability are removed), experts have difficulty in agreeing upon the primary cause(s) of serious diagnostic error. Although this was a small study, it suggests that sentinel event tracking and the current injury prevention models that rely on root cause analysis might not be well suited for dealing with errors of diagnosis. The causal web leading to patient injury from errors in diagnosis may, in fact, be too complex to be deciphered using traditional methodologies.

While this pilot study failed to find a reliable means to classify diagnostic errors, it did confirm the ability of clinical experts to reliably identify serious diagnostic errors when they are made. Greater awareness of diagnostic errors may serve as a stimulus for others to pursue research in this area. Without a reproducible system to categorize diagnostic errors, there cannot be an effective means of studying the diversity and magnitude of the problem. The “art” of clinical diagnosis should no longer be considered beyond quality of care scrutiny. The medical profession needs to take an active role in studying the underlying causes of diagnostic error.