Advancing the research agenda for diagnostic error reduction
- 1Department of Public and Occupational Health, EMGO Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands
- 2Division of General Medicine, Brigham and Women's Hospital
- 3Harvard Medical School, Boston, Massachusetts, USA
- 4Houston VA Health Services Research and Development Center of Excellence, Michael E. DeBakey Veterans Affairs Medical Center, Houston, Texas, USA
- 5Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
- Correspondence to Dr Laura Zwaan, Department of Public and Occupational Health, EMGO Institute, VU University Medical Center, Van der Boechorststraat 7, Amsterdam 1081 BT, The Netherlands;
- Received 8 April 2013
- Revised 28 June 2013
- Accepted 2 July 2013
- Published Online First 13 August 2013
Diagnostic errors remain an underemphasised and understudied area of patient safety research. We briefly summarise the methods that have been used to conduct research on epidemiology, contributing factors and interventions related to diagnostic error and outline directions for future research. Research methods that have studied epidemiology of diagnostic error provide some estimate on diagnostic error rates. However, there appears to be a large variability in the reported rates due to the heterogeneity of definitions and study methods used. Thus, future methods should focus on obtaining more precise estimates in different settings of care. This would lay the foundation for measuring error rates over time to evaluate improvements. Research methods have studied contributing factors for diagnostic error in both naturalistic and experimental settings. Both approaches have revealed important and complementary information. Newer conceptual models from outside healthcare are needed to advance the depth and rigour of analysis of systems and cognitive insights of causes of error. While the literature has suggested many potentially fruitful interventions for reducing diagnostic errors, most have not been systematically evaluated and/or widely implemented in practice. Research is needed to study promising intervention areas such as enhanced patient involvement in diagnosis, improving diagnosis through the use of electronic tools and identification and reduction of specific diagnostic process ‘pitfalls’ (eg, failure to conduct appropriate diagnostic evaluation of a breast lump after a ‘normal’ mammogram). The last decade of research on diagnostic error has made promising steps and laid a foundation for more rigorous methods to advance the field.
Diagnosis is one of the most important tasks a physician performs and determines subsequent treatment and patient outcomes. Although most patients can expect to be diagnosed correctly, diagnostic errors are being increasingly demonstrated in the patient safety literature.1–5 Recent research shows that diagnostic errors can often lead to severe consequences and preventable morbidity and mortality.2 ,6 ,7 Despite an urgency to study and reduce diagnostic errors, for various reasons discussed in other articles in the supplement, they have remained an underemphasised and understudied area of patient safety research.8 ,9
Over the past 5 years, the annual Diagnostic Error in Medicine (DEM) conferences have highlighted emerging research in this area, which we summarise here. For the purposes of this paper, we classify current research in this field into three main topic areas: (1) epidemiology of diagnostic error (frequency, types, detection methods); (2) causes of diagnostic error (cognitive and system issues) and (3) error prevention strategies (development, implementation and evaluation of interventions). This paper briefly summarises the methods that have been used to conduct research in these three areas, then outlines future research needs to better understand and reduce diagnostic error in medicine.
Research methods to study the epidemiology of diagnostic error
Existing research methods
Multiple research methods, largely retrospective, have been used to examine the epidemiology of diagnostic error (for an extensive overview on incidence, see Graber's article in this issue)10 Commonly used data sources include reports of malpractice claims,6 ,11 chart reviews of selected diagnoses or hospital admissions2 ,12 and autopsy reports.13 ,14 While these studies have given us a general appreciation of the burden of the problem, these estimates of incidence of the problem are far from precise or generalisable. Retrospective methods also introduce hindsight bias in judgments about error determination. Nevertheless, one advantage of retrospective methods is the potential availability of longitudinal data that spans the continuum of care of the patient (ie, outpatient visits, multiple subsequent hospital admissions, outcomes, etc). This allows researchers to retrospectively track the evolution of the diagnostic process over time and link it to diagnostic outcomes (ie, final diagnosis).2
Prospective methods have advantages such as reduced hindsight bias but used infrequently to study the epidemiology of diagnostic error. One example is the use of ‘standardised patients’ who present to providers in routine clinical settings without revealing their actual purpose.15 ,16 One study involved 23 rheumatologists who were visited by a standardised patient with known psoriatic arthritis. The diagnosis was missed or wrong in nine visits (39%). However, such methods are resource intensive. Other types of prospective methods involve presentation of hypothetical cases to assess physician cognitive error rates.17 ,18 However, these methods are mainly used to determine how varying circumstances or characteristics of physicians or patients influence error rates, and do not help determine incidence rates representative of a certain population.
Although previous studies have given us a general appreciation of the burden of diagnostic error, the estimated incidence rates of diagnostic errors vary greatly, from about 1% to as high as 55% in certain diseases or patient groups.1 This large variation can in part be explained by four factors:
Setting of the error and patient population: errors in certain specialty or ancillary care settings, such as radiology, appear to be less common than errors in clinical medicine. Also, diagnostic errors are more likely to occur with older or more complex patients compared with younger patients who have fewer comorbidities.1
Study methods: higher error rates appear to be related in part to use of data sources that over-represent severe or lethal outcomes, such as autopsy studies and malpractice claims. Conversely, other data sources, such as chart reviews, might not uncover all diagnostic errors due to lack of documented information (ie, ascertainment bias).
Type of diagnosis: certain diagnoses are likely more easily missed or confused, especially at an earlier stage of the illness, for example, tuberculosis or HIV.
Definition of diagnostic error: some studies define diagnostic error as any diagnosis that is wrong, delayed or missed, regardless of whether an error in the diagnostic process occurred.19 ,20 Other studies define diagnostic errors only when there is clear evidence of ‘missed opportunities’ for earlier diagnosis in a patient's care.12 ,21 Therefore, the rates of diagnostic error found in different studies are likely to vary substantially according to criteria used to define ‘error’.
Directions for future research on the epidemiology of diagnostic error
Obtaining more precise rates of error is an enormous challenge, given the heterogeneous data and methodological and logistical challenges noted above. Nevertheless, such an undertaking is necessary to establish base rates in common settings of care (primary care, general medical hospitals, etc), to both attract needed attention to the urgency and magnitude of the problem as well as prioritise and evaluate intervention efforts. Such a project could have an important impact in ways similar to the IOM's landmark report on medical errors and patient safety,22 which propelled the field of patient safety forward. Suggestions for specific methodological and research approaches to advance this agenda include:
Synthesise existing studies to estimate rates of diagnostic error: the true incidence of diagnostic error remains unknown. As more sophisticated methods for detecting diagnostic error are being developed, an early first step would be to apply rigorous methods to synthesise findings across studies in order to estimate incidence rates of diagnostic error. We recommend that incidence rates of diagnostic errors should be determined separately for the different settings (eg, inpatient hospital setting, primary care and outpatient setting) to account for differences in patient populations and diagnostic techniques. A synthesis of existing studies would also help further clarify the effect of methodological differences among the studies.
Evaluate incidence rates over time: longitudinal studies are needed to determine trends related to the incidence of diagnostic error, to identify any fluctuations in types of diagnostic errors in different settings and to understand what types of diagnoses might be more or less vulnerable to diagnostic error over time (eg, in response to the adoption of new diagnostic procedures and technologies). In addition, within organisations and systems that are implementing strategies to reduce error, it is important to know whether general patient safety interventions (such as improved safety culture, patient engagement), or certain targeted interventions (such as techniques to improve test result follow-up) actually result in lower rates of diagnostic error. Conducting studies using similar definitions, measurement methods and time intervals (eg, every 5 years) can provide insights into which processes, settings, patient groups or diseases are in greatest need of attention over time, and which interventions appear to have the greatest impact on patient safety.
The use of Electronic Health Records (EHRs) to identify diagnostic error: currently, manual chart reviews are considered the gold standard for detecting diagnostic errors. This method is labour-intensive and extensive resources are required.23 Several recent studies illustrate the use of EHR-based algorithms to identify patient charts with a higher probability of a diagnostic error.4 ,7 ,24 Such methods should now be explored further and refined for use in other institutions. Additional methods using more sophisticated algorithms should also be developed to proactively identify diagnostic errors before patient harm. This technique could also lend itself to study causes of error as well as facilitate tracking of error rates over time and testing the impact of interventions.
Causes of diagnostic error
Currently used research methods
Because many aspects of the diagnostic process are not easily captured using current study methods, the causes of diagnostic error are difficult to identify. Ideally, the diagnostic process is studied prospectively in a real clinical setting, but this poses substantial practical difficulties (eg, ethical issues, time constraints, measuring clinician thought processes unobtrusively, observer bias). Consequently, most studies conducted in naturalistic settings have examined the causes of diagnostic error retrospectively (eg, through record review which sometimes is followed by interviews).2 ,20 ,21 ,25 These studies have revealed vulnerabilities in clinical workflow processes or organisational issues that contribute to error (eg, policies that are ambiguous about who is responsible for test results follow-up).26 They have also yielded illustrative patterns of cognitive failures or biases, (ie, errors in the reasoning process of the physician),19 as well as the complexity of interaction between system and provider factors.25 Prospective evaluations have been generally restricted to artificial settings (eg, case vignette studies)27 and offer a complementary cognitive perspective.
Prior studies have revealed that the diagnostic step of ‘information synthesis’ was particularly prone to error.1 ,4 ,19 ,20 Retrospective studies conducted in naturalistic settings can identify contributory factors and types of diagnostic breakdowns7; however, they are seldom able to reveal the root causes of diagnostic error due to limited information about causality. For instance, in reviewing an error case in which a diagnosis was missed despite the availability of laboratory results to support the diagnosis, it may be difficult or impossible to know whether the physician lacked the appropriate knowledge to arrive at a correct diagnosis, or was influenced by a cognitive bias, distraction or faulty heuristic, or simply did not have time or access to the result.18 ,28 ,29
Prospective studies in controlled settings could be potentially useful to examine causal relationships. Experimental designs, in particular, can isolate the effects of certain biases or the mechanisms underlying the reasoning process.18 ,30 However, the next step for any such line of research would be to better understand how problems detected in the laboratory translate into real-world behaviours and problems. A key will be enabling a ‘culture of safety’ that will permit clinicians to honestly reflect on errors they have been involved in, and examine and test ways they could be prevented in their daily work in the future.
Directions for future research on causes of diagnostic error
Linking naturalistic and experimental studies: studies on the causes of error in naturalistic settings and experimental (case vignette-based, simulation-based, etc.) clinical research studies each carry distinct advantages and limitations. To maximise the knowledge gained from these disparate methods, we believe that more can be done to encourage linkages between research paradigms. The results of naturalistic studies could inform the hypotheses of experimental studies (eg, to better understand causal factors), and in turn, findings from these experimental studies could be tested for their applicability and validity in a naturalistic setting. For example, frequently missed lesions in x-ray evaluations that have been found in studies on malpractice claims could be studied in a clinical research setting (eg, by studying the eye-movements of radiologists when reviewing the x-rays). By replicating these errors in the clinical research setting, we can study why they occur and how the process could be improved. ‘Triangulating’ methodological approaches in this manner could accelerate our understanding of the pathogenesis and prevention of diagnostic error.
Testing newer models to study diagnostic error: newer models can be used to integrate and align the objectives of naturalistic observational studies and experimental studies. Advanced techniques for ‘root cause analysis’ developed for accident investigation outside of medicine have barely begun to be used in healthcare and offer promising returns. Another potentially fruitful avenue of research is application of the ‘situational awareness’ model, which has been widely studied in the human factors literature, specifically in the field of aviation.31 It can be applied to examine how practice environments and systems influence physicians’ cognitive processes, helping or hindering their ability to differentiate information relevant to diagnosis while maintaining a broader general overview of the patient.32 Situational awareness emphasises clinicians’ understanding of ‘what's going on’ and ‘what's likely to happen next’ as key elements in effective diagnostic decision making.33
Current intervention research
A recent systematic review evaluated the effects of patient safety strategies that focused on diagnostic error and found that only few strategies had an effect in terms of diagnostic error reduction or reduction of patient harm.34 While this review identified over 100 studies that tested the effects of interventions on the diagnostic process, two other recent narrative reviews showed that there are a large number of interventions that have been developed to reduce diagnostic errors, but have not been yet tested for their effectiveness. Specifically, these reviews evaluated the existing literature on outcomes of system-related and cognitive interventions.35 ,36 System-related interventions are those that are focused on addressing organisational process errors, and vulnerabilities, whereas cognitive interventions focus on improving physicians’ perceptual and thought processes (eg, educational interventions, cognitive de-biasing strategies; see related papers in supplement). Whereas many of the suggested interventions were judged to have great potential, only few were actually tested and implemented in practice. This paucity of published outcome studies underscores the urgent need to not only conceptualise and develop interventions but to systematically evaluate their effectiveness.
Directions for future research to improve diagnosis
1. Patient engagement: strategies to promote shared decision making and patient engagement in healthcare hold great promise for reducing errors and error-related harm.37–39 The patient is key stakeholder at every step in the diagnostic process and serves as the main source of information during the diagnostic process.40 The patient also has access to information that can help the physicians maintain and regain situation awareness. For example, patients may support physicians by accurately informing them in a timely fashion about changes in their clinical situation. Patients are uniquely suited to detect and report errors that occur during their care processes. Thus patients are an enormous and virtually untapped resource for reducing diagnostic error;41–43 research studies that examine this potential are few. Therefore, studies are warranted to examine ways in which patients can influence their physicians’ decisions by providing more accurate or focused information, asking questions, questioning diagnoses and discussing medical tests and diagnoses they may be concerned about.44 To be able to involve the patient in their medical process, the physician's role is changing as well. The physician needs to facilitate patient involvement by creating an environment in which patients can contribute, without shifting responsibility for accurate diagnosis to the patient. Specifically, the physician needs to create opportunities for patient involvement, for example, provide the possibility for patients to upload the patient history (eg, family history) online in advance as well as creating an atmosphere in which the patient feels comfortable to ask questions.
Furthermore, research should explore the potential diagnostic benefits of patients’ critical review of their own medical record.45
2. EHRs: EHRs are now being developed and implemented in many countries, and the focus on improving the diagnostic process through the use of EHRs is timely. Research has already shown that the EHRs can be used for detection of diagnostic error,5 ,7 and the next step is to apply these methods to the implementation and evaluation of interventions to improve the diagnostic process. Schiff and Bates created a conceptual model and made several suggestions for preventing or minimising diagnostic error with use of EHRs and other health information technologies.46 Some of the most promising suggestions are:
Improving access to information. An EHR provides longitudinal information about the patient's health status, including prior outpatient visits and hospitalisations. Although the ready availability of this information should facilitate the diagnostic process, research is needed to understand how best to facilitate display and use of this information to improve the diagnostic process. In particular, it will be necessary to identify and correct unintended consequences of EHR adoption, including ‘information overload.’47 Incorporation of tools in the EHR to facilitate diagnosis. Checklists have been shown to offer great potential for reducing errors in general48 ,49 and can be applied to the diagnostic process.50 Although there is limited evidence thus far to support the use of checklists for diagnostic error reduction,46 ,51 the EHR provides new opportunities to incorporate checklists into clinical practice to facilitate the diagnostic reasoning process.
Exploring newer forms of Health Information technology (HIT): New information technologies, such as the IBM supercomputer Watson, have potential to improve the diagnostic process52 by using natural language processing. Natural language processing allows systematic analysis of human natural language into medical concepts, which eventually can result in a shortlist of differential diagnoses. Although we are far from broad implementation of such techniques in clinical practice, research is needed to evaluate which techniques have the potential to reduce error. An extensive review of ways to reduce diagnostic error using HIT is described in the article of El-Kareh et al in this special issue.
3. Delineation of specific diagnostic process pitfalls: research on the frequency and causes of errors could reveal specific areas of improvement that can be targeted through focused interventions. For example, a commonly seen error in malpractice claims for missed or delayed breast cancer diagnosis is failure to pursue the appropriate additional diagnostic evaluation for a woman with a breast lump after her diagnostic mammogram returns as ‘negative’.6 Rather than recognising the imperfect sensitivity of this test in the face of a palpable mass and proceeding to do the recommended next test (eg, ultrasound-guided aspiration cytology), physicians have on occasion instead reassured the patient about the ‘normal’ mammogram and stopped the diagnostic workup.53 Attributing rectal bleeding to ‘hemorrhoids’ in a patient at high risk for colorectal cancer, rather than conducting additional diagnostic testing to rule it out, illustrates another common diagnostic pitfall.12 Specific pitfalls that contribute to greater-than-expected burden of diagnostic errors could be targeted through training and system redesign.
In conclusion, research on diagnostic error, although still nascent, has evolved significantly as compared with the state of knowledge a decade ago. In this paper, we highlight areas where research methods could propel the field forward in promising directions related to epidemiology, contributory factors and interventions. The recently founded Society of Improving Diagnosis in Medicine and the annual Diagnostic Error in Medicine conferences are platforms to bring together a multidisciplinary group of people (including clinicians, patients, researchers and educators) to advance the research agenda. This agenda could also be integrated with research agendas of other organisations (eg, National Institutes of Health, specialty societies as well as national research agendas, eg, the UK's large research agenda on cancer). Furthermore, funding in this area should be stimulated to operationalise this research agenda. Using well coordinated, multidisciplinary approaches, and with appropriate research support, the foundation laid thus far can lead to future methodological advancement in the field and to reduction of diagnostic errors.
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or any other funding agency.
Contributors All authors listed have contributed sufficiently to the project to be included as authors and approved of the final version.
Competing interests Dr Singh is supported by an NIH K23 career development award (K23CA125585), the VA National Center of Patient Safety, Agency for Health Care Research and Quality (R18HS017820), and in part by the Houston VA HSR&D Center of Excellence (HFP90-020). These sources had no role in the preparation, review, or approval of the manuscript.
Provenance and peer review Commissioned; externally peer reviewed.
this is an open access article distributed in accordance with the creative commons attribution non commercial (cc by-nc 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. see: http://creativecommons.org/licenses/by-nc/3.0/