Article Text
PDF
Abstract
Background Incorrect, delayed and missed diagnoses can contribute to significant adverse health outcomes. Intervention options have proliferated in recent years necessitating an update to McDonald et al’s 2013 systematic review of interventions to reduce diagnostic error.
Objectives (1) To describe the types of published interventions for reducing diagnostic error that have been evaluated in terms of an objective patient outcome; (2) to assess the risk of bias in the included interventions and perform a sensitivity analysis of the findings; and (3) to determine the effectiveness of included interventions with respect to their intervention type.
Methods MEDLINE, CINAHL and the Cochrane Database of Systematic Reviews were searched from 1 January 2012 to 31 December 2019. Publications were included if they delivered patient-related outcomes relating to diagnostic accuracy, management outcomes and/or morbidity and mortality. The interventions in each included study were categorised and analysed using the six intervention types described by McDonald et al (technique, technology-based system interventions, educational interventions, personnel changes, structured process changes and additional review methods).
Results Twenty studies met the inclusion criteria. Eighteen of the 20 included studies (including three randomised controlled trials (RCTs)) demonstrated improvements in objective patient outcomes following the intervention. These three RCTs individually evaluated a technique-based intervention, a technology-based system intervention and a structured process change. The inclusion or exclusion of two higher risk of bias studies did not affect the results.
Conclusion Technique-based interventions, technology-based system interventions and structured process changes have been the most studied interventions over the time period of this review and hence are seen to be effective in reducing diagnostic error. However, more high-quality RCTs are required, particularly evaluating educational interventions and personnel changes, to demonstrate the value of these interventions in diverse settings.
- diagnostic errors
- healthcare quality improvement
- performance measures
- patient-centred care
- patient safety
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. The data that supports the findings of this study are available in the tables and supplementary files of this article.
Statistics from Altmetric.com
- diagnostic errors
- healthcare quality improvement
- performance measures
- patient-centred care
- patient safety
Introduction
Medical error: importance, prevalence and the current state of the literature
Medical error can occur from initial diagnosis through to treatment.1 Most of this error is potentially preventable and derives from technical error, error in diagnosis, failure to prevent injury or errors in the use of a drug. In the USA, it is estimated that 49–79 medical errors occur per 1000 inpatient admissions.2 An economic analysis3 of the prevalence and burden of medication error in the UK also estimated that approximately 66 million potentially clinically significant medication errors were made each year. This error rate was found to be comparable with other health settings such as the EU and the USA.3 Furthermore, a 2019 meta-analysis4 found a 6% pooled prevalence of preventable patient harm within healthcare systems around the world, with error in diagnosis accounting for 16% of this preventable patient harm. The gravity of occurrence of diagnostic error has been emphasised in a UK study,5 which found that 65% of patients diagnosed with community-acquired pneumonia did not meet criteria for the illness. Overall, diagnostic error is defined by The National Academies of Sciences, Engineering, and Medicine as ‘the failure to (a) establish an accurate and timely explanation of the patient’s health problem(s) or (b) communicate that explanation to the patient’.6
Interventions to address diagnostic error
Reflecting the varied situations and contexts in which diagnostic error can occur,4 5 a wide array of interventions to prevent or minimise their impacts have been tested. These interventions range from active, point-of-care tools, such as checklists, digital diagnostic algorithms and increased patient reviews, through to reflective, system-wide measures such as incident reporting systems and chart reviews.7–9 However, the literature has a somewhat narrow focus, that is, testing a single context-specific strategy. While this may be useful information for some end users, it does not offer an accurate and comprehensive assessment of the effectiveness of the full variety of potential options available.
One paper that sought to take a comprehensive approach is McDonald et al’s 2013 systematic review10 of diagnostic error-reducing interventions, which addressed patient-related outcomes. The review also incorporated data from two previous narrative reviews, the first by Singh et al,11 which assessed system-related interventions addressing organisational vulnerabilities, and the second by Graber et al,12 which analysed cognitive interventions that could affect diagnosis. McDonald et al’s review10 identified six broad categories of intervention types targeted at reducing diagnostic error: technique (changes in equipment, procedures and clinical approaches), technology-based system interventions (system level technology-based tools), educational Interventions (educational strategies and training curricula), personnel changes (additional healthcare members and replacing certain professionals), structured process changes (feedback loops or additional stages in the diagnostic pathway) and additional review methods (additional independent reviews of test results).10 Additional review methods made up most of the total strategies (28%), while personnel changes contributed the least (4%). Of the 14 randomised controlled trials (RCTs), only three did not have statistically significant improvements in diagnostic error.10
The need for another systematic review
Since the publication of the McDonald et al10 review in 2013, the authors could find no other comprehensive review of the breadth of interventions targeted at reducing diagnostic error. During this time, the technology available and used by medical providers such as greater uptake of the internet, wireless technology and increasing processing power have changed dramatically.13 14 While a wealth of information has been written about a wide array of strategies targeted at reducing diagnostic error,11 12 these papers often restrict their topics into silos of individual interventions.15–34 Therefore, there is a need to collate and examine the currently available interventions aimed to reduce diagnostic error within healthcare settings. An updated review is likely to be useful to administrators for comparing the most recent ideas that take advantage of any advancements. In addition, an updated review that includes a sensitivity analysis of studies with a low risk of bias would allow clinicians and hospital administrators to access the strongest and most up to date information for decision making about potential interventions to minimise diagnostic error. This review aims to update and build on the McDonald et al review10 and specifically:
Describe the types of published interventions for reducing diagnostic error that have been evaluated in terms of a patient outcome since 2012.
Assess the risk of bias in the included interventions and perform a sensitivity analysis of the findings.
Determine the effectiveness of included interventions with respect to their intervention type.
Methods
Data sources and search terms
This systematic review largely replicates the methodology of McDonald et al’s10 2013 review. The search terms are identical (provided in online supplemental file 1); however, three databases were searched in this review (rather than two10) between 1 January 2012 and 31 December 2019. These databases were MEDLINE,35 the Cumulative Index of Nursing and Allied Health Literature (CINAHL)36 and the Cochrane Database of Systematic Reviews.37
Supplemental material
The search strategies detailed in McDonald et al’s study10 were followed to search the three aforementioned databases (see online supplemental file 1). The major Medical Subject Heading terms were ‘diagnostic errors’ and ‘delayed diagnosis’. Other search terms included (but were not limited to) ‘intervention studies’, ‘controlled clinical trial’, ‘treatment outcome’, ‘clinical competence’, ‘decision making’, ‘decision support techniques’, ‘medical records systems, computerized’, ‘patient participation’, ‘forms and records control/standards’, ‘physician-patient relations’, ‘reminder systems’ and ‘systems analysis’.
The search was limited to articles written in English and carried out with human subjects. This yielded a total of 1487 publications, as shown in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow chart38 (figure 1), plus another 387 relevant publications obtained from reviewing the references listed in systematic reviews identified from the search (reference review).
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow chart.38
Study selection
Once the qualifying publications were obtained, they were exported and uploaded to Covidence,39 and the duplicates were removed. A set of inclusion and exclusion criteria was formulated (provided in table 1 further) and used to screen literature based on the selection criteria used in McDonald et al’s systematic review.10 The articles were first screened via abstract and title, and the remaining qualifying articles underwent a full-text screen. The screening procedure was performed in two teams of two members, and the study period was divided in half so each pair had approximately the same number of articles to review. A small subsection of papers (10%) was randomly chosen to act as a calibration tool within the groups, and then between the groups to ensure selection bias would not occur within or between the assessing groups. Any disagreement was resolved by discussion. Publications were included if they delivered patient-related outcomes relating to diagnostic accuracy, management outcomes and/or information regarding patient morbidity and mortality. To qualify as relevant, a study required at least one of the aforementioned patient outcomes, and in order to be omitted, at least one exclusion criteria needed to apply. The full breakdown of how many publications were excluded, and the reasons for exclusion are provided in the PRISMA flow chart38 in figure 1.
The inclusion and exclusion criteria used to screen literature10
Data extraction and quality assessment
Data extracted from the 20 applicable publications15–34 included the study design, patient and clinician information, the type of diagnostic error being addressed, the experimental intervention as well as the results. The interventions from these studies were categorised based on the six categories of intervention types found in McDonald et al’s10 systematic review: technique, technology-based system interventions, educational interventions, personnel changes, structured process changes and additional review methods. No additional categories were needed.
Each article15–34 was analysed for their risk of bias (found in online supplemental file 2) using two Cochrane tools of assessment: the Risk of Bias 2 (RoB 2)40 tool for evaluating randomised studies and the Risk Of Bias In Non-Randomised Studies - of Interventions I (ROBINS-I)41 tool for evaluating non-randomised studies. These40 41 have been used to perform a sensitivity analysis of the findings.
Supplemental material
Results
Twenty studies15–34 met the review inclusion and exclusion criteria (shown in figure 1). These studies can be categorised under the six intervention types used and defined by McDonald et al,10 as shown in table 2 below:
Categories of interventions to reduce diagnostic errors10
Of the 20 studies,15–34 four were RCTs18 24 32 33 and 16 were non-randomised studies.15–17 19–23 25–31 34 The risk of bias of each RCT was determined using Cochrane’s RoB 240 tool (provided in online supplemental file 2). It was concluded that three18 24 33 of the four RCTs18 24 32 33 were of ‘low’ risk of Bias, and one RCT32 had ‘some concerns’. For the non-randomised studies, Cochrane’s ROBINS-I41 tool (provided in online supplemental file 2) was used to conclude that one study23 was of ‘low’ risk, 14 studies15–17 20–22 25–31 34 were of ‘moderate’ risk and one study19 was of ‘severe’ risk of bias. The distribution of the 20 total studies15–34 by their intervention type and year of publication is displayed in online supplemental file 3, and the data extracted from these studies are provided in table 3.
Supplemental material
Data extraction of included studies by intervention type
Technique
Eight studies15–22 investigated interventions related to medical technique. All interventions were implemented to alter a clinical approach and/or procedure (eg, the introduction of a treatment adjunct for differentiated thyroid cancer that also assists in the diagnosis of distant cancer metastases15). The majority of these studies,15–20 22 including one RCT,18 concluded that such interventions are associated with increased likelihood of accurate diagnosis. Time to diagnosis was also addressed by Pare et al,20 where emergency physician focused cardiac ultrasound was implemented in the emergency department. One article,21 however, found an increased rate of misdiagnosis following implementation of their intervention, spectrophotometric intracutaneous analysis, for skin cancer detection.
Technology-based system interventions
Five studies23–27 used either electronic health record (EHR) screening, or electronic alerting systems (eg, flagging patients who had not received follow-up for a suspected malignancy24 25) with the purpose of prompt physician action. These studies suggest a reduction in diagnostic error with the use of electronic methods. Three studies,24–26 including an RCT,24 employed an automated method to detect and trigger an alert for diagnostic delay based on information from EHRs. Another intervention23 focused on a specific diagnosis, with an EHR screening phenotype identifying patients with late-stage dementia on hospital admission. A ‘Natural Language Processing’ based computer algorithm was also implemented to identify key words in an electronic medical record (EMR) search, compared with a manual record search to diagnose asthma.27 All technology-based interventions23–27 resulted in a decrease in either diagnostic delay or missed diagnoses.
Educational interventions
Two studies28 29 used educational interventions to target missed diagnoses (of 13 ‘high-risk’ medical conditions28), misdiagnosis (diagnosis of stroke vs stroke mimics29) and delayed management (time from stroke code call to stroke team assessment, imaging and reperfusion therapy29). Both articles28 29 found that such interventions reduced the likelihood of diagnostic error. In one study, a clinical documentation improvement programme was designed to avoid missed diagnoses using educational conferences and reference cards of high-risk diagnoses available to resident doctors, and the revision of EMR note template prompts to assist in documentation.28 Meanwhile, the retrospective cohort review29 found that the implementation and revisions to an institution-developed in-hospital stroke code protocol significantly decreased the overall rate of stroke mimics. This protocol included education on stroke recognition and treatment algorithms, ultimately resulting in a reduction of delayed stroke assessment and improving treatment times.
Personnel changes
Two retrospective cohort studies30 31 individually examined the diagnostic effects of introducing additional healthcare professionals to a patient’s care30 and substituting healthcare professionals with another type.31 In one study,30 diagnostic delay of ankylosing spondylitis was reduced when hospital inpatients were assessed by a multidisciplinary team. Rates of missed diagnosis were also reduced with personnel changes, as evidenced at a tertiary care centre31 where a new triage strategy using emergency department physicians for ECG review was compared with the previous strategy of a nurse-led computer-assisted ECG review. As a result of the intervention, time to treatment with balloon catheterisation improved.
Structured process changes
Two RCTs32 33 addressed interventions that involved alterations to diagnostic processes. Ely and Graber32 investigated the merit of providing diagnostic checklists containing differential diagnoses for 63 commonly presenting symptoms to primary care physicians. The mean error rate for physicians who used checklists was not significantly different compared with those who did not. The effect of delay in reporting diagnoses was addressed in another RCT.33 Patients were randomised to either an immediate reporting service for musculoskeletal trauma in the ED, where the X-ray report was returned to ED at the same time as the patient or a delayed X-ray reporting group (usual care). This study33 concluded that the process change of immediate diagnostic reporting resulted in less ED interpretation errors and thus reduced misdiagnosis.
Additional review methods
One study34 addressed the impact of an additional review step in the diagnostic process. Intradepartmental double reading of histopathology specimens demonstrated a positive effect on the diagnostic process, indicating that a second review is worthwhile in terms of patient safety and quality of care.
Sensitivity analysis
The results of the RCT32 that demonstrated ‘some concerns’ and the non-randomised study19 assessed to have a ‘severe’ risk of bias were compared with the results of the 18 comparatively lower risk articles.15–18 20–31 33 34 All interventions evaluated in the lower risk studies were successful in reducing diagnostic error, except for one technique-based intervention21 that found an increase in skin cancer misdiagnosis. In comparison, the RCT with ‘some concerns’ for risk of bias32 was unsuccessful in demonstrating a significantly different diagnostic error rate following a structured process change. However, the study having a ‘severe’ risk of bias19 analysed a technique-based intervention that showed improved diagnostic accuracy of intrauterine fetal MRI scans compared with perinatal postmortem MRI scans for identifying fetal brain malformations.
From this sensitivity analysis, it can be concluded that the inclusion of higher risk of bias studies19 32 does not alter or skew the overall results of this review. This may be due to the very limited number (n=2) of higher risk studies found. Hence, the results of the primary analysis remain robust.
Discussion and conclusion
Comparison of our results to relevant past literature
This review differs from McDonald et al’s10 2013 review in its approach to risk of bias. Non-randomised studies with a high risk of bias and RCTs with a moderate or higher risk of bias were scrutinised in this review through a sensitivity analysis to ensure a high level of confidence in the results. In contrast, McDonald et al10 discussed all studies in their results, regardless of their risk of bias, without accounting for the effects of higher risk articles on their outcomes. Of their 109 studies,10 24 contained more than one applicable intervention type, whereas this review yielded studies that fit specifically under one of each of the six intervention categories.
McDonald et al’s 2013 review10 followed the narrative reviews published by Singh et al11 in 2011 and Graber et al12 in 2012. Singh et al11 and Graber et al12 individually examined studies specifically investigating system-related (n=43) and cognitive (n=141) interventions, respectively. Neither review11 12 conducted a risk of bias assessment of the analysed literature. Additionally, unlike this review, publications evaluating simulated interventions in artificial settings and theoretical interventions not tested in actual practice were also included.11 12 After excluding literature on theoretical interventions, the Singh et al11 and Graber et al12 reviews only contained 6 and 42 studies each that tested system-related and cognitive interventions, respectively.
In McDonald et al’s review,10 only three of the five low risk of bias RCTs demonstrated improvements in patient outcomes following the intervention. These three studies individually evaluated a technology-based system intervention (an electronic data collection form with feedback for abdominal pain interpretation), an educational intervention (teaching parents an acute illness observation scale to detect their child’s illness) and a personnel change (using nurse practitioners in providing care in the ED instead of junior doctors). The other two low risk of bias RCTs with unsuccessful interventions investigated a technique-based intervention and a structured process change. In contrast, all three low risk of bias RCTs18 24 33 identified in this review were successful in improving patient outcomes. These individually evaluated a technique-based intervention,18 a technology-based system intervention24 and a structured process change.33 Hence, the results of McDonald et al’s10 and this review commonly identify technology-based system interventions as the most promising intervention category in enhancing diagnostic performance and patient outcomes.
McDonald et al’s review10 broadly identified additional review methods as the most popular strategy aimed at minimising diagnostic error, whereas this review identified technique-based changes as the most common intervention type. When combining the results of McDonald et al’s10 and this review, educational interventions and personnel changes have been the least studied interventions, with only 13 and 8 publications on each, respectively, since 1966, while additional review methods have been analysed the most with a total of 39 studies. Over time, a shift in the focus of published interventions can be observed. Evaluation of additional review methods, the most studied intervention category10 from 1966 to 2012 (n=38), has steeply declined with only one study34 meeting our inclusion criteria. Meanwhile, from approximately 2001 to the end of McDonald et al’s review10 in 2012, an increase in technique-related interventions was reported. Since 2012, this growth in technique-related interventions has continued. Furthermore, McDonald et al’s10 and this review have consistently found technology-based system interventions as the second most common intervention category.
McDonald et al’s10 and this review also share many similarities between the nature of the successful interventions assessed under each category. In both reviews, most technology-based system interventions reduced misdiagnosis or prevented delayed diagnosis by applying patient screening methods on EHR. Like this review, most of the effective strategies relating to personnel changes in McDonald et al’s study10 involved adding more medical staff to the care team. The addition of another clinician to review the diagnostic process was also the most successful specific intervention under additional review methods in both reviews.
Differences in the nature of the successful interventions under each category are also evident when comparing McDonald et al’s10 and this review. The studies reporting effective educational interventions in this review only involved medical staff.28 29 In contrast, McDonald et al’s10 successful educational interventions included medical staff as well as patients and their families. In terms of structured process changes, this review found one publication33 showing significantly improved diagnostic accuracy following the immediate reporting of diagnoses. On the contrary, most of the efficacious structured process changes identified by McDonald et al10 included the addition of a tool such as a checklist.
In Singh et al’s review,11 all six articles testing system-related interventions in practice (eg, a picture archiving and communications system for trauma diagnosis) were effective in reducing diagnostic error, whereas the 42 tested cognitive interventions reported by Graber et al12 produced mixed outcomes overall. Unfortunately, many interventions deemed successful by Graber et al12 were simulated in artificial settings. Consequently, these results may be difficult to extrapolate in the real world. By excluding studies assessing simulated and theoretical interventions, this review overcomes this barrier and presents more practical and feasible diagnostic error-reducing strategies.
Limitations, strengths and implications
A clear limitation of this study is the narrow timeframe of 2012–2019 (8 years) for published literature to be reviewed. Consequently, our search yielded a comparatively smaller number of articles (n=20)15–34 that fitted our inclusion and exclusion criteria. We recognise this to be a small number of articles to provide a substantial comment on the overall nature of objective patient outcomes from recent interventions aimed at reducing diagnostic error. Furthermore, only four RCTs18 24 32 33 were identified, most interventions were not tested in more than one site and many studies were small. The included studies15–34 examined diagnosis for a variety of target conditions, ranging from typically non-acute outpatient issues (eg, skin cancer18 21 22) to highly morbid time-sensitive conditions (eg, acute aortic dissection20). However, it is also noteworthy that the acuity of a condition and the location in which it is usually diagnosed may influence the type of intervention chosen to improve diagnosis. Therefore, while the results of this systematic review highlight the trends in the research of diagnostic error and patient outcomes over the study period, it can only provide data on the specific interventions identified, and more research is required to demonstrate the value of diagnostic error-reducing strategies in diverse settings.
A limitation of the existing literature is the lack of a consistent approach to the measurement and/or analysis of diagnostic error.10–12 This problem is similar to that reported by McDonald et al,10 Singh et al11 and Graber et al12 where the issue of diversity in intervention types and bias in published literature towards ‘positive’ results make general recommendations for strategies to minimise diagnostic error difficult to provide. Furthermore, the included studies15–34 provided no information about the cost of implementing the intervention (eg, costs associated with new equipment or training the staff). As such, the practicality and feasibility of these interventions is difficult to determine and may be a barrier to their widespread implementation. Other examples of such real-world barriers include access to adequate training as well as patient adherence and participation in the proposed intervention. Strikingly, in the articles examined in McDonald et al,10 Singh et al,11 Graber et al’s12 and in this review, mortality and morbidity end points were seldom reported, thus forming another limitation when analysing the patient outcomes for all reviews. Therefore, more high-quality research is still required to advance the field of diagnostic accuracy. This may be achieved through increased funding and the implementation of different methodologies such as a critical realist methodology to analyse underlying mechanisms of why and in what contexts interventions work.
Given the dispersed nature of the tested interventions assessing medical error,10–12 the major strength of this review is its robust approach to gathering and presenting much needed recent information on the current interventions available for reducing diagnostic error and improving patient outcomes.15–34 These interventions have been tested in the real world and report measurable diagnostic outcomes. They also cover a range of specialties, settings and patient populations allowing for a broader application of the evaluated strategies into practice. With this information, medical professionals can make educated decisions about choosing to continue their standard medical practice or adopting an appropriate intervention to positively affect patients. Ultimately, this review hopes to inspire further research in the field of medical error so that no patient unnecessarily faces adverse outcomes.
Conclusion
Technique-based interventions (eg, teledermoscopy instead of clinical teleconsultations18), technology-based system interventions (eg, applying an electronic trigger to medical records to flag patients with suspected malignancies24) and structured process changes (eg, immediately reporting diagnoses of musculoskeletal injuries in the emergency department33) have been evaluated in high-quality RCTs and are among the most studied interventions since 2012. These categories therefore seem to be most effective in enhancing diagnostic performance. Educational interventions and personnel changes have not yet benefited from being sufficiently analysed and require further evaluation. Despite the increasing prevalence of studies assessing diagnostic error-reducing strategies, there is still a distinct lack of low risk of bias RCTs in this field.10–12 In particular, studies reporting patient morbidity and mortality outcomes of such interventions are scarce and require further research.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. The data that supports the findings of this study are available in the tables and supplementary files of this article.
Ethics statements
Patient consent for publication
Acknowledgments
The authors gratefully acknowledge the support and advice of our prior supervisor and thank Debbie Booth, Medical Librarian, for assistance with database search of literature.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors Each author’s contribution has been outlined further, as per the CRediT (Contribution Roles Taxonomy) classification: ND: conceptualisation (equal); data curation (lead); formal analysis (equal); investigation (equal); methodology (equal); project administration (lead); supervision (lead); validation (lead); visualisation (lead); and writing – original draft preparation (lead); writing – review and editing (lead). SB: conceptualisation (equal); data curation (supporting); formal analysis (equal); investigation (equal); methodology (equal); validation (equal); writing – original draft preparation (equal); and writing – review and editing (equal). CM: conceptualisation (equal); investigation (equal), methodology (equal); validation (equal); writing – original draft preparation (equal); and writing – review and editing (supporting). SH: conceptualisation (equal); investigation (equal); methodology (equal); validation (equal); and writing – original draft preparation (supporting). CLP: conceptualisation (equal); methodology (equal); supervision (supporting); validation (equal); and writing – review and editing (equal). All authors provide final approval of the manuscript and supporting files to be published. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.