Article Text
PDF
Abstract
Background Flaws in clinical reasoning are present in most diagnostic errors and occur even when physicians have enough knowledge to solve the problem. Deliberate reflection has been shown to improve diagnoses. The sources of faulty reasoning and how reflection counteracts them remain largely unknown.
Objective To explore the causes of faulty reasoning and the mechanisms through which reflection neutralises them by investigating the influence of salient distracting clinical features on diagnostic decision-making.
Design and setting In a prior study, 34 internal medicine residents and 50 medical students of the Erasmus Medical Centre, Rotterdam, diagnosed four clinical cases by means of non-analytical reasoning and four by reflective reasoning. In the secondary analysis of the data presented here, five internists independently evaluated the diagnoses and examined the nature of the diagnostic errors in relation to case features that gave rise to these errors.
Main outcomes Frequency of incorrect diagnoses caused by salient distracting features made through reflective and non-analytical reasoning.
Results Among residents, reflective reasoning (Mean diagnostic accuracy score (M)=2.09, 95% CI 1.77 to 2.40) led to a significantly higher number of correct diagnoses than non-analytical reasoning (M=1.71, 95% CI 1.37 to 2.04; p=0.03). This higher diagnostic accuracy was associated with fewer incorrect diagnoses triggered by salient distracting clinical features (M=0.47, 95% CI 0.26 to 0.68) compared with non-analytical reasoning (M=0.85, 95% CI 0.59 to 1.11; p=0.02). Students did not benefit from reflection to improve diagnoses.
Conclusion Salient features in a case tend to attract physicians' attention and may misdirect diagnostic reasoning when they turn out to be unrelated to the problem, causing errors. Reflection helps by enabling physicians to overcome the influence of distracting features. The lack of effect for students suggests that this is only possible when there is enough knowledge to recognise which features discriminate between alternative diagnoses.
- Decision-making
- diagnostic errors
- cognitive biases
- medical education
- continuing education
- continuing professional development
Statistics from Altmetric.com
- Decision-making
- diagnostic errors
- cognitive biases
- medical education
- continuing education
- continuing professional development
Introduction
The diagnostic decisions of physicians strongly affect patient outcomes. The medical literature has extensively discussed how diagnostic errors derived from flaws in physicians' reasoning can be minimised.1–3 Reflective reasoning has been shown to improve diagnostic decisions, at least in non-routine cases.4 5 However, neither the causes of diagnostic errors nor the mechanisms through which reflection minimises them are entirely clear. This study investigated both issues by exploring whether and how flaws in recognition and interpretation of clinical features caused by salient distracting cues in patients' history affect diagnostic decision-making.
Diagnostic errors occur in every specialty,6 7 and although many of these errors may be corrected in time or produce minor adverse effects, there are serious consequences for a significant proportion.6 8 9 Most diagnostic errors derive from flaws in physicians' cognitive processes, usually caused by faulty reasoning rather than by knowledge gaps. For instance, such flaws were implicated in 74% of the diagnostic errors in internal medicine in a study in American hospitals.3
Why would a physician fall prey to such flaws despite having the knowledge to arrive at a correct diagnosis? These cognitive diagnostic errors have been frequently associated with non-analytical diagnostic reasoning.1 2 Experienced doctors tend to generate diagnostic hypotheses through ‘pattern recognition’, that is, fast recognition of similarities between the case at hand and examples of previous patients or prototypical scripts of diseases that they have stored in memory.10–12 Pattern recognition is rapid, effortless and leads to appropriate decisions in most situations.13 However, as it occurs largely unconsciously, mistakes, when made, are seldom corrected.4 5 14–18 Experimental studies have suggested that a more reflective reasoning approach may prevent mistakes.4 5 15 19 For example, for residents in internal medicine a structured reflective approach to diagnosing complex clinical cases led to higher diagnostic accuracy compared with non-analytical reasoning.5 Moreover, in another study reflection counteracted cognitive biases occurring under conditions of non-analytical reasoning, leading doctors to ‘repair’ wrong initial diagnoses. This indicates they did have the required knowledge to solve the problem, although this knowledge apparently was not used to make the first diagnosis.15
Little is known about the mechanisms through which reflection minimises flaws in clinical reasoning. Such flaws have been frequently attributed to biases emerging from the use of heuristics (ie, rules of thumb) by physicians to make diagnoses.1 2 16–18 Attributing errors to bias explains what was wrong in the physician's reasoning. However, it does not explain why the physician engaged in faulty reasoning despite having the knowledge to solve the problem and it does not clarify how reflection brings the correct diagnosis to mind.
Since pattern recognition in non-analytical reasoning is triggered by cues in the patient's history,11 13 faults in recognition and/or interpretation of features present in a case might play a role in reasoning errors. We hypothesise here that difficulties may arise from the presence of salient distracting cues in a case. Such cues are findings that may grab physicians' attention, within the context of that case, because they are very suggestive of another disease that seems plausible at first sight, but is not the correct diagnosis. For example, in a patient with hyperthyroidism whose main complaints include muscle weakness in both limbs, a history of pravastatin use may lead a physician to immediately consider the diagnosis of statin-associated myopathy. This may lead to mistakes when a more careful analysis of the case that would allow recognising other findings and, eventually, the correct diagnosis, does not take place. Research into human perception has suggested that attention tends to be preferentially directed to perceptually salient features of a stimulus, even when these are irrelevant for the task at hand.20–22 Using perceptual saliency as an inspiration, we hypothesise that in clinical cases some (not necessary perceptual) features in a patient's history are also more salient than others for a physician who knows about the clinical significance of certain findings. Such salient features would also tend to attract physicians' attention immediately, triggering pattern recognition. Reasoning may be misdirected, leading to errors, when these features, despite being salient, are in fact unrelated to the present problem. Deliberate reflection on the case would presumably lead the physician to re-evaluate the salient cues and consider less salient cues as well, which would activate other diagnostic hypotheses, increasing the probability that the correct diagnosis is among them. Reflection would only help, however, when enough knowledge is available to recognise the relevance of other, potentially less salient cues.
We explored these ideas by undertaking a detailed, in-depth analysis of data collected in a previous study that established the beneficial effects of reflection on diagnostic accuracy.5 In that study, internal medicine residents and medical students diagnosed clinical cases under three different experimental conditions in a within-subjects design. Here, we will conduct a secondary analysis of these data to address our hypothesis about the role of saliency of features by providing a detailed analysis of the type of errors made under two of these conditions: a non-analytical and a reflective reasoning condition.
Method
Participants
Participants were 34 internal medicine residents (mean age=28.97; SD=2.25) of the Erasmus Medical Centre and 50 fourth-year students (mean age=24.00; SD=1.71) of the Faculty of Medicine, Erasmus University Rotterdam.5 The undergraduate programme has a 6-year curriculum, with the two final years dedicated to clinical clerkships. Participation was voluntary, and volunteers to the sessions received book vouchers in return.5
Materials and procedure
Participants diagnosed 12 clinical cases, each case consisting of a description of a patient's medical history, signs and symptoms, and tests results. An example of a case is presented in online appendix 1. All cases were based on real patients with a confirmed diagnosis and had been used in previous studies with medical residents.4 23 Online appendix 2 presents the list of cases used in the present study.
A within-subjects design was used. Participants diagnosed the cases by following instructions that led to one of three different reasoning conditions: non-analytical reasoning, reflective reasoning and ‘deliberation without attention’.5 They diagnosed four cases under each condition. This study analysed the cases that were solved under the non-analytical and the reflective reasoning conditions.i The cases were presented randomly in a booklet, and the sequence in which the conditions appeared in the booklet was also randomised (ie, all participants diagnosed four cases under non-analytical/reflective reasoning, but which cases were diagnosed under which condition differed among participants as did the order of conditions). In the non-analytical reasoning condition, participants were asked to read the case and write down the most likely diagnosis, doing their best to provide an accurate diagnosis as quickly as possible. Accuracy and speed were emphasised. Participants were informed that they were requested to perform a second task (solving a word puzzle of medical terms) after diagnosing each case. The researcher explained that they had to complete both tasks (ie, diagnosing the case and solving the word puzzle), and that they would only have sufficient time for the second task (solving the word puzzle) if they diagnosed the case quickly. Time spent to diagnose the case and to solve the puzzle was not recorded, but all participants completed the two tasks. Although some degree of reflective reasoning might always take place, this procedure has been successfully used in other studies to induce a more non-analytical reasoning approach, minimising chances that participants engage in elaborate analysis of the case under the non-analytical reasoning condition.12 15 24 In the reflective reasoning condition, participants were requested to read the case, indicate an initial diagnosis, and then follow instructions inducing an elaborate analysis of case information: they indicated which cues from the case corroborated or refuted their initial hypothesis, wrote down alternative hypotheses and proceeded with a similar analysis for each alternative diagnosis. After this analysis, they indicated the likelihood of the different hypotheses and made their final decision on the most likely diagnosis for the case. Time allocated to work on each case (including the diagnosis and the second task in the non-analytical reasoning condition) was equal (8 min) in all conditions.5
Analysis
Five board-certified internists affiliated with the Erasmus Medical Centre, all with more than 15 years of clinical experience, evaluated the diagnoses, without being aware of the condition under which they had been made, and the clinical features in the cases. The internists evaluated the cases independently, discussing the cases with each other only after completing the evaluation, by following a two-step procedure.
Diagnostic accuracy
All the diagnoses provided by the participants were classified, according to their accuracy, as ‘correct’, ‘plausible though incorrect’ or ‘entirely incorrect’. A diagnosis was considered ‘correct’ when the core diagnosis (as mentioned above, all cases had a confirmed diagnosis) was cited by the participant. For example, ‘pancreatitis’ was judged as ‘correct’ in the case presented in online appendix 1. When the diagnosis was not the correct one but could reasonably be considered as part of the differential diagnosis, it was classified as ‘plausible though incorrect’, for example, ‘cholangitis’ in the above-mentioned case. Finally, when the participant's response did not fall into one of these categories, it was classified as ‘entirely incorrect’ (eg, ‘peptic ulcer’ or ‘hepatic encephalopathy’ in the same case). Agreement was reached in 88% of the diagnoses, and disagreements were resolved by discussion.
Identifying salient distracting features and diagnoses associated with them
The experts analysed each case description to identify salient distracting features, that is, findings that tend to call physicians' attention, within the context of that specific patient's clinical presentation, because they are very suggestive of another disease that may seem plausible at first glance. However, despite their potential to catch attention, these features are in fact unrelated to the patient's present problem and may lead physicians to a wrong path of clinical reasoning and thereby to incorrect diagnoses. Examples of salient distracting features and the wrong diagnoses triggered by these features in the case of acute alcoholic pancreatitis are shown in online appendix 1. The experts independently listed the salient distracting features present in each case and the erroneous diagnoses that they could trigger. They agreed on 82% of the features/diagnoses, and subsequent discussion resolved divergent views.
Based on the results of this second step, the diagnoses previously evaluated as entirely incorrect were then classified as ‘entirely incorrect triggered by salient features’ and ‘entirely incorrect due to other sources’. The number of diagnoses in each category (‘correct’, ‘plausible though incorrect’, ‘entirely incorrect triggered by salient features’, ‘entirely incorrect due to other sources’) made by participants under the non-analytical and the reflective reasoning condition were computed. We performed a repeated-measures multivariate analysis of variance (MANOVA) with reasoning mode (non-analytical reasoning vs reflective reasoning) as a within-subjects factor and the number of correct diagnoses, the number of plausible diagnoses, the number of incorrect diagnoses triggered by salient distracting features, and the number of other incorrect diagnoses as outcome variables for the group of residents. A second similar MANOVA was performed for the group of students. Subsequently, separate univariate analyses of variance were performed on each of the outcome variables. The significance level was set at p<0.05 for all comparisons (two-tailed). SPSS V.15.0 for Windows was used for the statistical analyses.
Results
Tables 1 and 2 show the diagnoses subdivided by category (ie, ‘correct’, ‘plausible though incorrect’, ‘entirely incorrect triggered by salient distracting features’, ‘entirely incorrect due to other sources’), made under non-analytical and reflective reasoning by residents and students, respectively.
Diagnoses made by residents (N=34) as a function of diagnosis type and reasoning mode, expressed as a proportion (total number of diagnoses in parentheses)
Diagnoses made by students (N=50) as a function of diagnosis type and reasoning mode, expressed as a proportion (total number of diagnoses in parentheses)
Among residents, a significant effect of reasoning mode on the number of diagnoses per category was found, F(3, 31)=2.95, p<0.05, Wilks' λ=0.78. As indicated by the univariate tests, the multivariate difference can be explained by a significant effect of reasoning mode on the number of correct diagnoses, F(1, 33)=5.46, p=0.03, and on the number of incorrect diagnoses triggered by salient distracting features, F(1, 33)=6.30, p=0.02. Reflection upon the case resulted in a significant increase in correct diagnoses compared with non-analytical reasoning due to a lower number of incorrect diagnoses triggered by salient distracting features. Comparisons between the other types of diagnoses did not reach significance (see table 1).
A different pattern emerged for the students. No significant multivariate difference was found, F(4, 46)=0.12, p=0.97, Wilks' λ =0.99, showing that reasoning mode did not affect how often they made each type of diagnosis, which can be confirmed by the results of separate univariate analyses (see table 1).
Discussion
We investigated whether cognitive diagnostic errors may be caused by difficulties in overcoming the influence of salient distracting cues present in a case. A panel of five experts followed a standardised procedure to analyse the diagnoses made by residents and students in a previous study, classifying their diagnoses according to their accuracy and identifying salient distracting features associated with the erroneous diagnoses. Findings suggest that the confounding influence of salient distracting features was responsible for a substantial proportion of the incorrect diagnoses when physicians solved the cases through non-analytical reasoning. Under this condition their reasoning was apparently misled by the sheer saliency of some of the features presented by the patient, such that they ended up with a wrong diagnosis triggered by these salient distracting cues. However, when deliberately reflecting upon cases, physicians were somehow ‘protected’ against this negative influence, as suggested by the lower number of diagnostic errors associated with salient distracting cues and, consequently, came up with a larger number of correct diagnoses. The students, however, did not benefit from reflection to overcome the influence of salient distracting cues.
Findings seem consistent with studies on saliency in perception research which show that attention tends to be directed, without the intention of the observer, towards certain features of a task20–22 either because they are perceptually salient or because they become salient to the individual due to his/her prior experience with similar problems.20 25 It is reasonable to suppose that some features in a clinical case, even when irrelevant to the present problem, would become salient for a physician who had previous patients with similar histories or when the clinical significance or potential seriousness of a certain finding is known.
Indeed, our findings suggest that some cues in a patient's history may be so salient that they immediately attract physicians' attention, activating a diagnostic hypothesis through pattern recognition. This similarity-based diagnostic approach will tend to fail when these clinical features, despite being salient, turn out to be unrelated to the patient's present problem. In contrast, deliberate reflection upon a case also guides the physician's attention towards other, less salient cues that are relevant to the problem, thereby activating scripts of other diseases, which generates other hypotheses. By verifying these alternative hypotheses against evidence from the case, physicians are able to distinguish between features that are in fact related to the problem and those that are not, either dismissing the salient cues as irrelevant or integrating them with other less salient relevant cues into a new, and eventually correct, diagnosis. Previous studies have shown that reflection may reduce errors, at least in non-routine cases,4 5 but the mechanisms through which reflection helps were not clear. Our findings suggest that one such mechanism may be that reflection directs physicians' attention to other, sometimes less salient features, thereby activating knowledge available in physicians' memory. This notion seems consistent with the studies showing that diagnostic errors produced by faulty clinical reasoning occur despite the existence of enough knowledge to diagnose the problem,3 8 and with research associating correct diagnoses with the discrimination of critical, relevant cues.26
Reflection allowed the residents to overcome the influence of salient distracting features, whereas it did not reduce the number of wrong diagnoses triggered by such features among students. Gaining from reflection, as the residents did, apparently depends on having in memory knowledge about the features that define a certain disease while making another unlikely. This enables the physician to recognise the correct diagnosis among alternative plausible diagnoses. Without having yet started their clinical clerkships, the students had very limited knowledge and clinical experience, and consequently, thinking further on the case was not sufficient for them to recognise less salient cues and alternative diagnoses.
One possible limitation of the study comes from the restricted experience of the physicians participating in the study. It is unclear whether more experienced practitioners would be equally prone to the distracting influence of salient features. However, it is known that reliance on pattern recognition and difficulties to revise an initial wrong diagnosis increase with experience,27 which makes the influence of salient features among older doctors rather more likely and potentially more serious. Furthermore, the findings are in line with our hypotheses, but the study does not provide direct evidence for the role of salient features in producing diagnostic errors, since its influence was established post hoc. An experimental approach would be required to demonstrate causal relationships. Indeed, the present study generated preliminary results and hypotheses that might be tested in future experimental studies. In such experiments, the saliency of clinical features in to-be-diagnosed cases could be manipulated in different ways, which would allow for determining how saliency affects diagnostic accuracy.
This study has cast some new light on the causes of cognitive diagnostic errors and the mechanisms by which reflection reduces such errors, which may contribute to the design of educational approaches for the development of students' and residents' diagnostic competence. The results suggest that flaws in clinical reasoning might derive from difficulties in overcoming the influence of cues that are salient but in fact unrelated to the problem. Deliberate reflection, however, seems to reduce mistakes.
Acknowledgments
The authors are grateful to Professor S W J Lamberts, Professor J P H Wilson, Dr A H van den Meiracker, and Dr E Zijlstra for their valuable contribution in the evaluation of the diagnoses and case features. They also thank Professor Geoff Norman for his suggestions during the data analysis.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Download Supplementary Data (PDF) - Manuscript file of format pdf
Footnotes
Competing interests None.
Ethics approval The study was approved by the Research Ethics Committee of the Department of Psychology, Erasmus University Rotterdam.
Provenance and peer review Not commissioned; externally peer reviewed.
↵i We excluded the third condition, ‘deliberation without attention’, from the present analysis, because we were interested in investigating the effects of reflection on diagnostic accuracy, and it is not possible to determine whether reflective or non-analytical approaches prevailed in diagnostic reasoning in that condition.