CDS-Dx work

The report by Sibbald et al published in this issue of BMJ Quality & Safety adds to the growing evidence that decision support can improve the diagnostic process.4 This study was conducted at six different schools, and asked trainees and staff in internal medicine or emergency medicine to diagnose a series of challenging cases using the ‘Isabel’ CDS-Dx platform either early (right after the chief complaint) or later (after reviewing all of the relevant clinical information) in the diagnostic process. Students and residents included more diseases in their consideration with early use of the ‘Isabel’ CDS-Dx platform, a finding that could be important in improving diagnosis education. The most dramatic finding in the study was that later use, which is likely how it will be used in practice, improved the likelihood that the correct diagnosis would be included in the differential by 8%. Moreover, physicians said that using the software was easy and relatively fast.

To put this result in context, the incidence of diagnostic error in primary care practice is roughly 10%; we get it right only 90% of the time. Even a 5% improvement, which would elevate us to 95%, would eliminate half of the harm associated with diagnostic error, thought to be the most serious safety concern in the USA today.

Diagnosis is likely the most complex cognitive tasks that humans face. There are, roughly, only 200 symptoms but over 10,000 diseases, and each disease may present in different ways, depending on the patient. Standard textbooks of medicine cover less than 1000 of these, and often describe just classic presentations. The current generation of CDS-Dx systems, including Isabel, cover thousands more, and CDS-Dx platforms that specialise in rare diseases are emerging.5 Rare diseases will affect roughly 6% of the population at some point in their lifetime, with conditions that most clinicians will have never seen, or even heard of. Another unique resource for diagnosis is the VisualDx CDS-Dx system, which includes over 45 000 images, as especially helpful resource for conditions with visible manifestations. These include the variable presentations of common entities, for example, how the same rash might appear in patients with different skin colour.

The positive results seen by Sibbald et al are no aberration. Just as examples, a study in family medicine found that use of DXplain for decision support improved diagnostic accuracy by 8% in simulated cases.6 Similarly, Kostopoulou et al reported an 8% absolute improvement in diagnostic accuracy in diagnosing a series of adult cases using a homegrown CDS-Dx resource.7 An important finding in this latter study was that using decision support did not lead to excessive testing or consultation, a logical concern whenever you start to consider a larger range of possible diagnoses. The AHRQ SAFER III systematic review analysed 31 published studies and concluded that CDS-Dx improved diagnosis,8 as did a separate review from the Margolis Center: CDS-Dx ‘has the potential to augment clinicians’ intelligence, support their decision-making processes, help them arrive at the correct diagnosis faster, reduce unnecessary testing and treatments otherwise resulting from misdiagnosis, and reduce pain and suffering by starting treatments earlier.’9 At a minimum, reviewing CDS-Dx suggestions ensures that ‘System 2’, the deliberate, conscious consideration of the diagnostic possibilities, will be available to complement our intuition.

CDS-Dx tools are not perfect by a long shot. The diagnostic recommendations will vary considerably depending on what key findings are entered, and the list of possibilities is long. This can be overwhelming for novices, but experienced physicians excel at sorting these out. And not all CDS-Dx products are created equal; substantial performance differences have been noted in comparative studies.10 A frequent criticism of CDS-Dx research studies is that they typically ask physicians to evaluate written case scenarios, a far cry from the world of real patients. The study of Kostopoulou et al described above addressed this by using standardised patients in real practice settings.7 Similarly, using Isabel in diagnosing admissions to a paediatric intensive care unit increased the correct diagnosis from 89% to 92%.11

Why is CDS-Dx underused?

Especially in view of their potential to improve diagnosis, usage of CDS-Dx systems to date is underwhelming.12 Even when they are available, clinicians most often do not consult them. Several factors may be in play (figure 1). The case for improving diagnosis has not reached the front lines. Most clinicians believe they are doing just fine, and do not appreciate the magnitude of the diagnostic error problem, or that they are susceptible to it. This is where quality assurance and performance improvement efforts could make a real difference, by providing the appropriate motivation to recognise and address diagnostic error as a major patient safety concern. The problem is not the technology, it lies in motivating users to take advantage of it. Even when they are aware of the general problem of diagnostic error, many clinicians believe they personally do not need any help, especially for cases which seem straightforward. Unfortunately, cases that seem simple end up compromising the bulk of cases involving diagnostic error; overconfidence is ubiquitous in diagnostic practice, and calibration is poor—clinicians do not do well in predicting whether their diagnosis is right or wrong.13

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Key reasons clinical decision support for diagnosis is underused.

Also, the ever-increasing pressure to increase clinical production creates an incentive to skip an extra step, like checking the recommendations from a CDS-Dx system. In his classic study of why CDS products in general are underused, Sittig et al found that 80% of clinicians said they were less likely to use CDS advice if they were behind schedule, and 84% said they were behind schedule some, most or all of the time.14

The future of CDS-Dx

There are three promising trends regarding the uptake and potential use of CDS-Dx systems:

First, never mind that clinicians are reluctant; patients are eager users of CDS-Dx tools. A variety of patient-facing versions are now available and usage is increasing impressively. As an example, patients can access a version of the Isabel CDS-Dx system customised for their use, and it provides triage advice in addition to diagnostic suggestions.15 All told, over 50 patient-facing apps that aim to improve diagnosis are findable on the internet,16 and according to surveys, patients like these tools.17 Patients will not be as adept as clinicians in deriving appropriate diagnostic considerations, but having patients offer up these alternatives to their physicians may prove to be an effective way to prevent the many diagnostic errors that result from premature closure and simply accepting the first possibility that comes to mind, assuming that their physicians are receptive and listening.

Second, knowledge resources are improving in parallel. In contrast to the spotty uptake and use of CDS-Dx systems, decision support resources that directly access medical knowledge content are widely deployed, and can independently help improve diagnosis.18 UptoDate, perhaps the most widely used content-focused CDS product, has been shown to improve diagnosis,19 and similarly positive results were seen when clinicians used a CDS system linked to BMJ Best Practice.20 It is reasonable to hope that as physicians acquire familiarity with using content-focused CDS systems, this may translate to increased acceptance and use of the CDS-Dx products that specialise in optimising differential diagnosis. Future products that marry the two may be the ideal answer.

Third, using decision support tools is becoming faster and easier. Consulting a CDS-Dx product inserts an additional, somewhat disruptive step in clinical workflow.21 Listening to the needs and wants of clinical users,22 and considering the human factors issues involved,23 can help optimise how often and how effectively these resources will be used. The next steps in the evolution of CDS-Dx, where these systems become well integrated in the electronic health record, will be especially important. Next-generation systems, based on machine learning and artificial intelligence, will scour the clinical notes to suggest diseases that should not be missed, and ‘push out’ diagnostic suggestions instead of our having to pull them.24 25