Main findings

The findings of this study identify the initiation and closure of the cognitive process in the clinical consultation as those most exposed to risk of diagnostic error. Initiation is a critical step as it sets the frame for subsequent information search, whereas closure occurs when thresholds for stopping the search have been met. We show that cognitive biases developed at framing appear to relate directly to errors at the end of the process. We refer to these as warning signs (table 2), as we believe they can be used to alert the clinician to the increased risk of diagnostic error. Previous studies4 have also highlighted these two steps as the points where most cognitive errors occur. However, our findings build on these, providing insights into the underlying sources of the biases that made the process go wrong. This is consistent with Wearth and Nemeth's17 observation that ‘[we] do not learn much by asking why the way a practitioner framed the problem turned out to be wrong. We do learn when we discover why that framing seemed so reasonable at the time’.

The initial process of framing a new case is mostly subconscious, and occurs within the domain of System 1 of the DTC (instant, fast response). The salient feature of the case leads to recognition of similar cases, and framing is modulated by constructs from experience, knowledge, values and social context.22 If there is no instant recognition, System 2 (slow, analytic thinking) may be engaged, but this may consist of no more than gut feelings.21 In the cases analysed here, there was a dominant focus on a single diagnosis. It is likely that other options were also entertained but forgotten by interviewees. The most significant biases occurring at this stage related to patients with pre-existing diagnostic labels and those with underlying psychosocial problems (table 1). Ignoring red flags or critical cues may have been related to processes being biased through the frame, but equally well, they could be explained by lack of knowledge of the significance of these clinical features.

We have previously suggested23 that the informal ‘rules’ that clinicians use to cease their search for further clinical features during a consultation (ie, stopping rules for diagnostic closure) involved three criteria: (1) high-risk conditions have been excluded and other options appropriately ruled in; (2) there was a direct response to the patient's needs; and (3) there was a reliable safety net in place. As a result of the biases we describe based on our current findings, it appears that in some cases red flags and critical cues suggesting alternative diagnoses were ignored or misinterpreted. The focus was on ruling in early diagnoses, rather than the usually preferred option of ruling hazardous conditions out first. This is consistent with System 1 being dominant, ignoring the reflective System 2. We found that ignoring or failing to search for important cues may be due to knowledge gaps or biased reasoning processes. Participants’ frequent reference to not being experienced suggests knowledge gaps contributing to a number of errors. For these reasons, and contrary to much of the literature (with the exception of Norman and Eva10), we therefore conclude that lack of knowledge is indeed likely to be an important factor in diagnostic error.

Finally, we found that participants did not use numerical values as thresholds for stopping the collection of more clinical data. We hypothesise that this relates to the complexity of these judgments: needing to satisfy decision rules based largely on subjective inputs. First, the clinician needs to take into account the basis of ruling disease in or out. Most early formulations are at least partly based on System 1 knowledge and have a tendency to bias unless one takes time for reflection through the influence of System 2. Robust data for predictive values may not exist or not be powerful enough to lead to safe closure. Second, assessing the values and individual needs of patients in the context of an illness is subjective. Third, confidence levels in a safety net may not be reliable, yet confidence in safe closure must be closely dependent on confidence in the quality of the net.23

If we add to these issues the impact of the context or environment on the clinician's decision-making24 and willingness or ability to involve System 2, it becomes apparent why Bayesian and similar ‘rational’ approaches are not the norm. It comes as no surprise when one of our GPs refers to closure as it feels like a nebulous thing.23 We believe that clinicians use individualised and often tacit guides for how to deal with the problem and other ‘multifarious factors that come into reckoning when making decisions.22 With this package they then make judgments, described by Polanyi16 writing about personal knowledge: in respect of choices made in the exercise of personal judgment …. there is always a range of discretion open in a choice…. In view of the unspecifiability of the particulars on which such a decision will be based, it is heavily affected by the participation of the person pouring himself into these particulars and may in fact represent a major feat of his originality, and concludes that valid choices can be made by submitting to one's own sense of responsibility. This last statement sums up the core of professionalism, where we expect a deep sense of responsibility from the decision maker.

So, what can we offer to reduce error? In an ideal world clinicians would have robust clinical data with high true positive and negative characteristics. This could then be used to develop decision aids, as suggested by Buntinx et al25 However, such information is scarce and there would still be a great deal of subjective judgment to be made. Providing indicators of early warning signs where errors may occur in the diagnostic process may help to prompt a reflective review of the cognitive process. We need to look at ways of presenting this as an integral part of the clinical process. This approach may then become part of the ongoing continual professional development for clinicians. Figure 1, based on our results is a summary of how warning signs may be incorporated in a practice environment to constructively promote reflection in practice.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Questions to relate potential sources of error to cognitive process: anatomy of diagnostic error.

Strengths and limitations of the study

We designed the study to be consistent with desired future directions as outlined in the introduction. It is based on a strong theoretical framework provided by the DTC. The strength of this model is not only that it ‘intuitively supports how our brains work’,12 but particularly that the initiation of the process is consistent with System 1 working at a subconscious level. Our interviews explore what is salient in a presentation26 and show how initial biases influence the diagnostic process. We identify warning signs on the way, alerting clinicians to risk of error.4 These may have the potential to reduce diagnostic error.11

Limitations relate to three main issues. First, sampling: participants were not chosen at random but consisted of self-selected individuals made up of more than half of the cohort we approached. This is an almost universal problem with studies that involve busy and senior clinicians. Our research needs to be replicated in other settings in the UK or in countries with different health systems, or with other specialties. However, given that our findings are consistent with other studies with different methodologies and use a strong theoretical base the results are likely to have a degree of generalisability. Second, hindsight bias: clinicians generally become aware of errors after an event and the information may be fragmentary. They construct a narrative based on hindsight, ending up with ‘illusions that one has understood the past’.15 In the construction of the narrative, hindsight bias converts events ‘into a coherent causal framework’.17 Occurring at a subconscious level, the bias is inevitable and all previous studies of error were tainted by it. We therefore attempted to minimise these limitations by examining the clinicians in real-life situations rather than a laboratory, and avoiding data that were collected for other reasons (eg, litigation),17 interviews tried to reconstruct the context in which error took place. Third, we do not have evidence that alerts such as the warning signs we propose will reduce diagnostic errors in clinical practice. Similarly, deliberate practice27 has not been shown to be successful in non-procedural specialities but in view of its success in procedural clinical settings, would be a strong contender to be explored.4

Implications for practice

There are two prerequisites for change. (1) Early and systematic recognition of errors including near misses: this could be achieved through regular, non-threatening, in-practice audits or significant event analyses. Without this it will not be possible to reduce hindsight bias. (2) Provision of a clinical environment that promotes reflection in action to detect the causes of process bias and knowledge gaps. This may be more feasible when working in a group of trusted peers and using methods such as incident reviews and journal meetings focused on recent errors as a way of reviewing the diagnostic reasoning processes as well as knowledge gaps. Since reflection will need to focus both on the cognitive process and evidence-based diagnosis resources, it would be important to have meetings facilitated by a trained, preferably internal, GP.

Implications for future research

We do not know if the practice changes that we propose will lead to better clinical care and this needs separate evaluation. To explore the generalisability of our findings we need to replicate this study in groups of clinicians in other specialties and settings. Future studies will need to deal with cases closer to the event and place even more emphasis on context. The model of deliberate practice27 is likely to be suitable for ongoing professional development and training to reduce error. Some of these have led to improved clinical care, albeit in interventional specialties, which suggests that similar improvements might be feasible in primary care, but this needs evaluation.