Article Text
PDF
Abstract
Background Diagnostic errors (missed, delayed or wrong diagnosis) have recently gained attention and are associated with significant preventable morbidity and mortality. The authors reviewed the recent literature and identified interventions that address system-related factors that contribute directly to diagnostic errors.
Methods The authors conducted a comprehensive search using multiple search strategies. First, they performed a PubMed search to identify articles exclusively related to diagnostic error or delay published in English between 2000 and 2009. They then sought papers from references in the initial dataset, searches of additional databases, and subject matter experts. Articles were included if they formally evaluated an intervention to prevent or reduce diagnostic error; however, papers were also included if interventions were suggested and not tested to inform the state of the science on the subject. Interventions were characterised according to the step in the diagnostic process they targeted: patient–provider encounter; performance and interpretation of diagnostic tests; follow-up and tracking of diagnostic information; subspecialty and referral-related issues; and patient-specific care-seeking and adherence processes.
Results 43 articles were identified for full review, of which six reported tested interventions and 37 contained suggestions for possible interventions. Empirical studies, although somewhat positive, were non-experimental or quasi-experimental and included a small number of clinicians or healthcare sites. Outcome measures in general were underdeveloped and varied markedly among studies, depending on the setting or step in the diagnostic process.
Conclusions Despite a number of suggested interventions in the literature, few empirical studies in the past decade have tested interventions to reduce diagnostic errors. Advancing the science of diagnostic error prevention will require more robust study designs and rigorous definitions of diagnostic processes and outcomes to measure intervention effects.
Statistics from Altmetric.com
Introduction
A growing body of evidence identifies diagnostic error (missed, delayed or incorrect diagnosis) as an important patient safety issue.1–5 Although not all of these errors translate into harm and many go undetected, a substantial number are associated with preventable morbidity and mortality.6 7 Diagnostic errors affect every medical discipline and all types of patients. However, the focus on diagnostic errors has lagged behind the rest of the patient safety movement.8 9
Studies have begun to identify the root causes that contribute to diagnostic error.10–12 These causes include either one or more of the following: clinician cognitive factors, system factors and patient factors. Cognitive factors include perceptual and thought processes, which are in turn influenced by differences in clinician training and experience, predisposition to cognitive and affective biases, fatigue, stress, and a variety of other elements. System factors refer to organisational vulnerability to diagnostic error and may include faulty communication practices, inadequate coordination of care, inadequate supervision, poorly designed technology and work environment, reduced availability of resources or personnel, inadequate feedback, and a culture that does not necessarily promote effective learning from errors.10 Patient-related factors include variability in communication styles and practices, heterogeneity in patients' clinical presentation to providers, and differential access to personal health information.13
Reducing the likelihood of diagnostic errors and error-related harm is a critical priority. Recent insights about diagnostic error aetiology have stimulated ideas about potential solutions. However, to our knowledge, strategies to reduce diagnostic errors have not been compiled or comprehensively reviewed since the release of the Institute of Medicine (IOM) report To Err is Human.14 We therefore conducted a literature review of articles published over the past decade to identify key interventions to reduce or prevent diagnostic errors. Our aim was to identify interventions that have been, or could be, implemented to address system-related factors that contribute directly to diagnostic errors. For the purposes of the review, we broadly categorised patient factors along with system factors. This paper examines system-related interventions, while a future companion paper reviews cognitive interventions (including decision support) to improve reliability of clinical reasoning.
Methods
Search strategy
We used multiple search strategies to identify candidate articles that describe interventions to prevent, reduce or mitigate diagnostic errors. Although many advances and interventions in healthcare may aim to improve diagnosis (eg, new diagnostic tests or screening methods), we focused specifically on system-level interventions to prevent or mitigate medical errors in the diagnostic process. To avoid searching potentially hundreds of thousands of indexed papers, we used more restrictive than inclusive strategies to select for diagnostic errors. Thus, we conducted a search of the PubMed database that combined the major medical subject headings (MeSHs) ‘Diagnostic Errors’ or ‘Delayed Diagnosis’ and one or more relevant MeSH terms and keywords to capture both system and cognitive interventions (see table 1 for complete list). We also focused our initial search on the time period after the release of the IOM report To Err is Human,14 2000–2009, to focus on more recent literature. We limited our search to English language publications that focused on humans and had abstracts that could be used for initial screening. This strategy yielded 949 articles.
Medical subject heading (MeSH) terms and keywords used as qualifiers for diagnostic errors or delayed diagnosis, in alphabetical order
We employed several secondary strategies to locate additional relevant articles for review. First, we manually reviewed the references of the articles we identified through PubMed as described above. Second, we used an internet-based search engine (Google) and searched topic-specific research databases (AHRQ's PSNet, PsycINFO, and the Air University Library Index to Military Periodicals) with a subset of search terms listed in table 1. Third, we identified additional recommended references from several authorities in the field of diagnostic error and decision-making sciences. Finally, we identified relevant articles released in 2010 after the cut-off date of the formal review but relevant to the topic. Together, these secondary search methods yielded an additional 160 articles.
Selection strategy
Because the field is nascent and evolving, we also reviewed the literature for intervention concepts that have been suggested by expert commentators, usually based on studies examining the epidemiology or aetiology of these errors. Thus, we considered two broad classes of articles: ‘actual’ interventions that were tested to reduce error or harm in medical diagnostic settings; and ‘suggested’ interventions, that is, those that had not been tested. The latter category was included to help refine our search for tested interventions, inform the state of the science, and highlight potential areas of future research. Articles that tested actual interventions discussed measurable changes in patient behaviours or organisational services, processes, systems, structures or products to prevent or mitigate diagnostic error. We incorporated all study designs, including review papers in the case of suggested interventions.
We excluded studies that described inter-rater or observer variation in the absence of an intervention; validations of screening instruments or tests; single case reports; assessments limited to provider satisfaction, preference or acceptance of interventions; and techniques to enhance diagnosis involving screening instruments, specific tests or technologies (eg, a newer generation CT scan). We also excluded studies of the development of risk models and reports of diagnostic error frequency or aetiology.
Abstracts were reviewed by three health services researchers and categorised as ‘included’, ‘excluded’ or ‘unsure’. To improve reliability and consistency of categorisation, the three reviewers first independently reviewed 20 abstracts, compared categorisation and refined their categorisation criteria. Two physicians with expertise in diagnostic error research further validated the inclusion/exclusion process by reviewing a random sample of excluded articles, and all articles categorised as ‘unsure’ and ‘include’, a strategy that helped achieve better inter-rater agreement. Disagreements in categorisation were resolved by team consensus.
Data extraction
We extracted data using structured data collection forms. For the studies reporting actual interventions, we documented study design, content and duration of intervention, type of intervention subjects, scope of intervention or ‘reach’ (ie, by number of participants), outcome measures and intervention effectiveness. For suggested interventions, we documented the focus (disease, condition, etc) and methods for diagnostic error prevention. All data collection forms were checked for completeness. All team members, including expert physicians, participated in the review and interpretation of results.
Categorisation
Diagnosis is a multistep process that depends on the functioning of the provider, patient and health system. Accordingly, we categorised system-based interventions according to five previously described, interactive steps13 of the diagnostic process: the patient–provider encounter, which involves clinician decision-making and test/referral ordering based on details of patient presentation; performance and interpretation of diagnostic tests; follow-up and tracking of diagnostic information over time; subspecialty and referral-specific issues; and patient-related care-seeking and adherence processes. Several interventions were not specific to a particular step in the diagnostic process and were categorised as ‘general interventions’ (table 2).
Taxonomy of diagnostic error dimensions
Results
We identified 43 articles on system-related interventions that met the criteria for full review. The majority of articles did not describe empirical studies, but rather provided suggestions for interventions based on the origins of diagnostic errors, observational research of system/patient factors, or promising results from studies of related topics (eg, patient satisfaction with test notification) (table 3). Six articles reported empirical outcomes of actual system interventions (table 4). These six studies were non-experimental or quasi-experimental and measured outcomes before and/or after an intervention among a small number of clinicians or healthcare sites. Measures of diagnostic error varied markedly between studies, depending on the setting and type of intervention and the diagnostic process involved. In the sections below, we summarise all of the 43 selected studies according to the five interactive steps of the diagnostic process.13
Proposed system-related intervention ideas to address multiple dimensions of diagnostic error
Studies that tested system interventions to address dimensions of diagnostic error
The patient–provider encounter
Two studies were related to diagnostic error during this step. In both studies, the goal of the intervention was to avoid delayed or missed diagnosis of traumatic injuries through changes in care processes. Perno et al15 described the implementation of a paediatric trauma response team, whereas Howard et al16 implemented a comprehensive re-evaluation of trauma patients within 24 h of admission. Both interventions produced positive results: implementing the paediatric response team significantly reduced delayed diagnosis of injury,15 and tertiary examination of trauma patients identified significantly more previously missed injuries.16
Diagnostic test performance and interpretation
One study tested an intervention to prevent diagnostic errors related to diagnostic test performance and interpretation. This trial, conducted in the emergency room setting, focused on the implementation of a Picture Archiving and Communications System (PACS), which electronically acquires, transfers and stores radiographical images.17 Using the PACS system improved diagnostic performance by reducing the overall misdiagnosis rate, although the rate of serious misdiagnosis did not change.
Follow-up and tracking
A number of papers focused on timely follow-up of test results, modes for follow-up, and outcomes. Of these, three described actual interventions: Singh et al24 examined the reliability of electronically communicating positive faecal occult blood test results in a system where over a third of results were not followed up. After identifying and correcting a software misconfiguration in the electronic health record (EHR) that prevented communication of test results to primary care providers, they found that timely follow-up increased significantly and was sustained at the fourth month following the intervention. Poon and colleagues designed Result Notification via Alphanumeric Pagers (ReNAP), an application that enables clinicians to indicate preferences for notification of patient-specific laboratory test results via an alphanumeric pager.25 ReNAP was well received, with 780 different clinicians using the feature within a 12-month period and usage averaging 2300 times per month. Piva and colleagues reported that a computerised test result notification system improved communication of critical laboratory values. Computerised notification was both faster (average of 11 min) and more successful (90% notification success rate within 1 h) compared with standard telephone notification only (average time of 30 min to notification, with less than 50% success within 1 h).26 Although these usage statistics implied an improvement in delivery of test results, in the latter two studies no actual data on follow-up of the delivered information was provided. Taken together, the studies illustrated the potential of technology and monitoring to improve transmission of important diagnostic information to clinicians, although no evidence of reduced diagnostic delays was provided.
Many articles suggested potential strategies to prevent test results from being lost to follow-up. Suggested system interventions included both processes to facilitate appropriate follow-up (eg, explicit communication policies for test results, highly structured hand-off procedures, and pre-planned follow-up for any diagnostic test) and structural changes such as use of electronic tracking systems and patient navigation programs.1 18 20–23 27 29 Hanna et al,19 for instance, described a broad intervention intended to facilitate improvement in communication of test results across multiple hospitals within Massachusetts. The Massachusetts Coalition for the Prevention of Medical Errors and the Massachusetts Hospital Association created a consensus group to identify the tests and the abnormal test results that should be considered critical and communicated in a timely manner, and the groups distributed this ‘starter set’ to hospitals statewide.
Referral-related issues
Although we did not find any tested interventions in this category, strategies to ensure availability of appropriate expertise have been suggested as interventions to reduce diagnostic error (eg, when there is no radiologist to read films overnight from the emergency room).29
Patient-related issues
The literature suggested several strategies to reduce error by better engaging and communicating with patients, although none of these were tested. Seven studies measured patient satisfaction and preferences with various methods of test result notification.31–37 Although not focused on diagnostic errors, two recently published literature reviews summarised the effectiveness and feasibility of patient engagement as a potential intervention for error prevention. Schwappach44 identified several key forces that promote patient engagement, including beliefs about self-efficacy, behavioural control, and the perceived ability to help prevent adverse incidents. Longtin et al45 concluded that, while patient engagement has been well documented in studies of decision-making and chronic disease management, patient participation in error prevention has not been explored. The authors provided a conceptual model of factors that influence patient participation in preventing errors.
Finally, other suggested interventions focused on improving patient education and communication between patients and providers to reduce errors. Two articles emphasised the need to anticipate patients' potentially faulty interpretations or reasoning during the diagnostic process.41 57 Another study suggested that better adherence to future care for abnormal Papanicolaou smears might result when adolescents visit a clinic prior to their follow-up colposcopy appointments.43 This literature demonstrates a need to consider the patient's perspective in designing interventions to reduce diagnostic errors.
General interventions (not specific to any step)
A number of articles suggested possible strategies to ameliorate ‘error-producing conditions’ that contribute to diagnostic error. Although we found no studies of actual interventions with these aims, many suggested interventions included structural and/or process changes to complement or improve providers' diagnostic performance. These included specific strategies such as second readings of key diagnostic tests, clinical decision support, and feedback to clinicians on their diagnosis (many of these will be discussed in detail in the companion paper),1 20 58 as well as general re-design of the working environment to produce better decision-making.59 Zwaan et al54 suggested methods to evaluate such interventions by measuring both the ‘suboptimal cognitive acts’ that could lead to diagnostic error (eg, not ordering a recommended test) and physicians' workload and fatigue at the time that they made the diagnosis.
Other publications suggested interventions to reduce diagnostic errors by learning from errors encountered locally. Schiff et al2 described the use of physician reports to identify and analyse diagnostic errors and suggested that organisations could identify potential preventive strategies through a similar process. Similarly, Colgan30 discussed the potential value in mandatory disclosure and review of all diagnostic errors encountered in a cohort of surgical and cytopathology cases. Articles also discussed the potential application of information systems to reduce diagnostic errors. Becich et al55 reviewed the opportunities for pathology informatics to enhance patient safety. Singh et al56 examined the range of potential communication breakdowns during the diagnostic process that can lead to error and identified opportunities for information technology to reduce these breakdowns. Finally, Schiff and Bates 22 focused on multiple ways in which EHRs can help in the prevention of diagnostic errors, provided they are designed and used appropriately.
Discussion
Our literature review of system-related interventions to reduce diagnostic error published in the past decade yielded very few empirical outcome studies. Because system-based interventions are favoured by many as the preferred approach for addressing diagnostic error, the results of our review are rather surprising.14 Our findings highlight a large gap between suggested interventions and those that have been operationalised and evaluated empirically. Many interventions suggested were already close to implementation, if not already underway, but lacked data to support their effectiveness in reducing diagnostic error. For instance, system-based interventions based on EHRs and health information technology have received a great deal of attention, but compelling studies are relatively few. Nevertheless, a handful of system interventions that were tested (eg, an electronic system to acquire, transfer and store radiographic images17 and process-of-care changes in emergency settings) demonstrated some degree of effectiveness in reducing diagnostic error.15 16 Interventions to promote more ‘patient-centred’ care (eg, empowering patients in their diagnostic process) represent another concept which, though broadly accepted, has not been tested as a means of reducing error.
Although patients constitute an important and largely neglected resource for improving outcomes related to diagnostic error, no empirical study found during our review examined the direct effect of patient-related interventions. For example, directly notifying patients of abnormal test results has been suggested as a reliable back-up process to help ensure that important results are not missed, but this has not been formally evaluated. An interesting example of how patients can be engaged in this context is the now mandatory reporting of all mammography reports directly to the patient.60
Our review has several implications for future interventions to reduce diagnostic error. Despite the high volume of care delivered in the primary care setting, few intervention studies directly addressed the primary care work system. The dearth of such studies was surprising because several intervention ideas for the primary care setting had been well conceptualised in the literature. These promising but as-yet untested strategies include improving follow-up and tracking of abnormal or critical test results, improving hand-off processes, systematic tracking of diagnostic errors and implementing rapid patient follow-up on certain high-risk initial diagnoses. Many of these are ripe for testing and implementation.
Advances in other areas of patient safety over the past decade have not been systematically applied to the science of diagnostic error reduction. In particular, one area we found to be largely absent from the literature was the science of human factors.61 To reduce mismatches between system-based interventions and the capabilities of providers and patients who interact with them, human factor principles should be applied to the design and development of future interventions. For instance, rapid prototyping techniques could be used to identify awkward and confusing interfaces, while testing the interventions in simulated or actual clinical settings might help identify unintended consequences.62 63 Better designs could help ensure that once an error occurs, it does not cascade through the entire multi-stage diagnostic process. Design of other health IT-based applications could also benefit from these same principles. For example, EHR-based intervention design must take into account the technology (software, hardware, content of data, information and knowledge, user interface) and the workflow in which it will be implemented, the people who use and implement it, the organisation in which it will be implemented, and the external legal and regulatory influences in play.64 Taking into account this interactive ‘socio-technical’ perspective will allow development of concomitant strategies to build resilience into the EHR work system and mitigate harm, if it occurs.65 Thus, the fields of cognitive science, informatics, human factors and engineering must come together to design some of these health IT interventions.
Testing and implementation of interventions other than IT to reduce diagnostic error in real-world practice will also need to take into account contextual factors that might affect their success.66 Factors such as policies and procedures, safety culture, organisational and teamwork-related factors could have a substantial impact on the effectiveness of systems-based interventions to reduce diagnostic errors. For example, implementing and testing a diagnostic error reporting system for physicians requires significant institutional commitment,67 and this might not be possible to obtain in many institutions. Recent evidence suggests that most of these contextual factors are generally not reported.66 Measurement and analysis of contextual factors that affect testing or implementation of interventions might be challenging, but it would provide others with useful information for applying these interventions to their own settings.66
Our review also highlights some of the main challenges in designing future interventions and studies to measure their impact. First, because of the multifaceted nature of these errors, and the fact that there are many other complex variables involved, the actual intervention effect might be difficult to demonstrate. Second, the impact of interventions on improving patient safety might be difficult to estimate because most studies did not specifically link errors or outcomes (such as delays) to adverse events. Although some robust methods to capture specific aspects of diagnostic errors, such as timeliness of diagnosis, have been used, these ‘process measures’ might not always link to reliable clinical outcomes. The few studies that did measure outcomes in terms of an actual diagnostic error rate focused on very specific clinical scenarios (eg, missed trauma injuries), measurement of which does not generalise broadly across care settings or disease conditions.15 16 In general, measurement science (definitions and rigorous process/outcome measures) in this area remains underdeveloped. Third, observational studies were most commonly used to measure outcomes before and after an intervention, with a small number of clinicians or healthcare sites, without a control group. Controlled study designs are desirable, but not always called for or practical. Fourth, although we categorised interventions in one of five process steps to account for system-related diagnostic processes, design and implementation of interventions to reduce diagnostic errors in real-world practice should also account for potential interaction between two or more of these steps.68 As evident in several studies that we could not categorise (general category), it is not always possible to categorise interventions according to these steps.
Our review had several limitations. Although distinguishing system-related from cognitive interventions facilitates understanding of diagnostic errors and discussion of possible interventions, we acknowledge that most diagnostic errors involve complex etiologies that are related to both system and cognitive performance.69 We could not delineate how the system-based interventions impacted providers' cognitive and perceptual capabilities. System-based interventions to facilitate clinical decision-making (eg, implementation of electronic clinical decision support systems) fall into this category and will be discussed in detail in the companion paper. We also used restrictive search criteria to identify literature specific to diagnostic errors or delays. As a result, we likely missed several key papers, especially when interventions were suggested in contexts that were not directly related to diagnostic error. Lastly, we focused largely on studies after the year 2000 in an attempt to capture progress made in the field in the past decade, but in doing so may have excluded earlier important work from our review.
In conclusion, our review summarises the state of the science in the design of future interventions to reduce diagnostic errors in healthcare. In light of the gaps in knowledge demonstrated in the recent literature, we believe that future studies should be multifaceted, focus on real-world clinical practice, and aim to measure the direct effects of interventions on rates of errors in diagnosis. Advancing the science of diagnostic error prevention will require more robust study designs and rigorous definitions of diagnostic processes and outcomes to measure intervention effects.
Acknowledgments
We thank Annie Bradford, PhD, for assistance with medical editing.
References
Footnotes
The authors of this paper are solely responsible for its content. The findings and interpretations in the paper do not represent the opinions or recommendations of the institutions with which the authors are affiliated, the Agency for Healthcare Research and Quality, the NIH or the US Department of Health and Human Services.
Funding This study was funded by the Agency for Healthcare Research and Quality (AHRQ) Task Order Contract No. HHSA290200600001, Task 8. Dr Singh is additionally supported by an NIH K23 career development award (K23CA125585), the VA National Center of Patient Safety, Agency for Health Care Research and Quality, and in part by the Houston VA HSR&D Center of Excellence (HFP90-020).
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.