Present limitations in the diagnostic error literature

Dave and colleagues’ review serves an important purpose by comprehensively updating the published literature around interventions to improve diagnosis. Yet, their synthesis also highlights the limitations in the diagnostic safety literature—among them, the variation in outcomes and inconsistent methodological quality—which limits the generalisability of the studies included in the review. Unfortunately, after another 8 years of research, practising clinicians and organisational leaders have gained little additional evidence about which interventions are effective at preventing diagnostic errors.

This lack of progress highlights a familiar issue in the patient safety literature, where interventions to preventing adverse events have often been slow to develop. Part of the explanation is that interventions to improve patient safety are by definition implemented in complex adaptive systems, with systems embedded within other systems, and response to any individual change difficult to predict. Interventions to improve safety must therefore be developed with a theory of change9—an overarching rationale that links an intervention to the specific outcomes, and specifies the contextual factors10 that influence the probability of success. The 2015 National Academies of Science and Medicine report Improving Diagnosis in Health Care stated that ‘The diagnostic process is a complex and collaborative activity that unfolds over time and occurs within the context of a health care work system.’ The complex nature of diagnosis heightens the importance of developing robust programme theories to inform intervention development, and the need to take the context of the intervention into account at both the development and implementation stages. This is particularly true for improving diagnosis, since the diagnostic process requires a high cognitive load for diagnosticians and takes place across varying clinical environments. The lack of attention to theory of change and healthcare system context in quality improvement studies has been identified as a problem for the patient safety field over the past decade11; as the diagnostic safety field continues to develop, those interested in improving diagnosis will need to avoid the mistakes of the past.

How should attention to theory of change and healthcare system context translate to intervention development? Interventions to help improve diagnosis for similar patients cannot be designed or implemented without understanding the social and cultural contextual factors in which a problem such as a delayed diagnosis took place. A case of delayed diagnosis of sepsis, based on similar cases at our hospitals, illustrates these and other factors that need to be addressed in future research to improve diagnosis (see box 1). A theory of change for improving diagnosis of sepsis would focus on identifying the primary barriers to accurate diagnosis in the emergency department, using qualitative methods. This would likely identify difficulty in communicating with patients with limited English proficiency, inability to obtain collateral information due to visitor restrictions and availability bias due to the COVID-19 surge as some of the major contributors to the delayed diagnosis. Interventions to address these barriers could include improving the availability of interpreters in the emergency department, facilitating video-conferencing with patients’ families (or being willing to liberalise morbidity and mortality conference based on this case). In other words, we would typically need a combination of the intervention types found in the systematic review by Dave and colleagues, to effectively prevent such a delayed diagnosis.

The role of context in the diagnostic process is especially important when planning interventions to improve care for vulnerable and marginalised populations. Many hospitals—including ours—already have technology-based system interventions (such as electronic health record (EHR) alerts based on physiological variables) to help clinicians identify patients with sepsis, with the goal of initiating evidence-based treatment in a timely fashion. There is some evidence in simulation studies that the use of clinician decision support systems (CDS) that generate a list of possible diagnoses can help improve diagnostic accuracy.12 However, prior research on the effectiveness of CDS at preventing other types of adverse events has shown that CDS typically have a small impact on the quality and safety of care. This is because CDS are often implemented without accounting for the social and technical context in which they will be used.13 It is possible—perhaps even likely—that widespread use of CDS will have similarly small effects on improving diagnostic accuracy if they are deployed without sufficient consideration of the complex environments in which clinicians make diagnoses.

Furthermore, as the US Agency for Healthcare Research and Quality Making Healthcare Safer III report noted, ‘The diagnoses generated by CDS tools are only as good as the information that is put into the system; if the initial assessment of the patient (eg, physical examination finding) is incorrect, the output is likely to be incorrect.’14 In the case described above, a language barrier and the inability of family to accompany the patient resulted in inadequate information being gathered during the initial clinician–patient interaction—which likely would have limited the ability of a CDS in this case to help generate the correct diagnosis. There is considerable interest in using ‘big data’ derived from EHRs to develop decision support algorithms to improve diagnosis, but concern has been raised that such artificial intelligence-based approaches may actually worsen disparities in diagnosis15 in part through the mechanisms illustrated in this case—clinical data on patients with language barriers or who receive care in resource-limited settings may be under-represented in EHR-based data sets, potentially leading to inaccuracies in diagnostic algorithms. The real world is more complex, where it is not just the individual clinicians’ cognitive processes that need to be targeted by a unimodal intervention like the CDS, but also the surrounding system context to improve diagnosis. Addressing inequities in diagnosis will therefore require additional qualitative work that engages patients and families in order to understand the context in which misdiagnosis occurs and inform the development of interventions that improve diagnosis for all patients.

Recommendations for future, real-world research

As the diagnostic safety field develops, researchers should be mindful of the challenges encountered in efforts to improve other safety problems, as learning lessons from prior efforts to improve safety will help inform interventions to improve diagnostic safety. Fundamentally, diagnosis will only be improved by recognising the complex sociotechnical nature of the diagnostic process, so the following suggested recommendations can be used to design and test future approaches (box 2).

First, researchers should plan studies using programme theories that inform interventions and how they will improve diagnosis. For example, the NASEM report outlines a conceptual model for the overall process of diagnosis; researchers developing interventions targeting specific aspects of this process (for example, communication of the diagnosis to the patient) should develop and report a programme theory for how their intervention will result in an improvement. The planning process should also include describing the healthcare system context in which they seek to improve diagnosis16 and incorporating key contextual features into the intervention design. Second, interventions should seek to target both the cognition and decision-making of individual clinicians, and the systems in which they work. At the moment, we may not have enough knowledge about which cognitive interventions and which systems-based interventions should be combined, but this should be the goal. For example, we may be close to being able to study the combination of an intervention that improves differential diagnoses and one that proactively alerts clinicians and provides guidance to evaluate abnormal test results that indicate important diagnoses (eg, new anaemia in an adult that may indicate colon cancer). The process of designing interventions must also engage patients and caregivers who can tell us about breakdowns that only they are aware of, as well as some of the implications of these breakdowns, to inform how to best target the intervention.17–19 Third, researchers should use the most rigorous study designs feasible,20 and interventions and their implementation should be described so they can be replicated.21 Studies should measure both diagnostic processes (ranging from patient recognition and communication of symptoms, to access to care, to clinician decision-making, testing and follow-up, to communication of diagnoses to patients, to helping patients understand and treat their diagnoses) and patient-level outcomes (correct diagnosis, correct treatment, harm and health status). Finally, we strongly recommend using both quantitative and qualitative methods to fully understand diagnostic improvement interventions in a real-world context. Traditional quantitative research is needed to measure the effectiveness of interventions, and qualitative and realist research approaches are needed to help understand how and why they work in complex systems.

This work will require more research funding from governments, foundations and perhaps the private sector, and will need to target a wide variety of diseases,22 sites of care and patient populations. Future progress should be assessed by both traditional systematic reviews such as those performed by Dave and colleagues, and by realist reviews that seek to determine what interventions work for which patients, under what conditions and how the interventions work.

The systematic review by Dave and colleagues is a signpost indicating the limitations of current approaches. The increased recent attention to improving diagnosis is overdue, but to achieve its potential, the diagnostic safety field will need a different approach to improving diagnosis—one that incorporates lessons from the efforts of the past.