Characteristics of included studies
| Author | Design | Participants | Experimental conditions | Outcome measure | Sample size | Findings |
|---|---|---|---|---|---|---|
| Educational | ||||||
| Hershberger, 199729 | Between-subjects | Medical students | (I/V) Seminar teaching on major biases. (C) No seminar | Evidence of heuristic use from performance on Inventory of Cognitive Bias in Medicine (ICBM), preintervention and postintervention | n=118 intervention, n=112 control | Intervention significantly improved performance on ICBM |
| Nendaz, 201145 | Between-subjects | Sixth-year medical students | (I/V) Case-based reasoning seminars with emphasis on explicit reasoning steps, self-reflection and cognitive psychology concepts. (C) Standard case-based reasoning seminars | Accuracy of final and differential diagnoses on two assessment cases | n=15 intervention, n=14 control | Intervention group performed better on differential diagnosis but not better on final accuracy of diagnosis |
| Reilly, 201328 | Pretest–post-test | Residents | (I/V) Three-part, 1-year curriculum on cognitive bias and diagnostic error, including lectures, group-based discussion, online interactive modules and video vignettes. (C) No curriculum | Performance on the Diagnostic error Knowledge Assessment Test (13 items) | n=38 intervention, n=42 control | Knowledge of cognitive biases and strategies significantly improved compared with baseline and control group |
| Round, 199946 | Between-subjects | Fourth-year medical students | (I/V) Seminar consisting of small group discussion followed by teaching on cognitive biases and Bayes Theorem. (C) No seminar | Performance on Diagnostic Thinking Inventory | n=84 intervention, n=102 control | Intervention group significantly outperformed control group on DTI |
| Schnapp, 201453 | Within-subjects | Residents | (I/V) 30 min lecture on 10 key diagnostic errors | Error identification on 10 clinical vignettes at 3-month follow-up | n=37 completed preintervention and postintervention | Significant improvement on only 3/10 error types |
| Sherbino, 201447 | Between-subjects | Senior medical students | (I/V) 90 min interactive seminar on cognitive forcing strategies. (C) No seminar | Evidence of heuristic use from instigation of search for secondary diagnosis on six clinical scenarios | n=145 intervention, n=46 control | Immediately after testing, <50% successfully show debiasing, vulnerable to false positives. No difference between intervention and control groups |
| Workplace | ||||||
| Checklist | ||||||
| Graber, 201460 | Within-subjects | Staff-level ER physicians | (I/V) Use of general and symptom-specific checklists on shift | Extensiveness of differential diagnosis on shifts, resource usage | n=15 completed preintervention and postintervention | Trends towards more items in differential and ordering of more tests and consults with checklist use compared with baseline, though not statistically significant |
| Shimizu, 201348 | Between-subjects | Senior medical students | (1) Instructions to interpret (C) intuitively, then (I/V) using general debiasing checklist. (2) Instructions to interpret (C) intuitively, then (I/V) using differential diagnosis checklist | Diagnostic accuracy on five clinical scenarios | n=91 debiasing checklist, n=91 differential diagnosis checklist | Differential diagnosis checklist significantly improves diagnostic accuracy over intuitive reasoning; no effect of general debiasing checklist |
| Sibbald, 201361 | Within-subjects | Cardiology fellows | (1) Verification with checklist. (2) Interpretation and verification with checklist. (3) Undirected interpretation. (4) Interpretation and verification without checklist. | Diagnostic accuracy on 18 ECGs | n=15 completed all conditions | Checklist use resulted in more correction of errors in verification conditions and fewer errors overall |
| Sibbald, 201354 | Pretest–post-test | Residents | (I/V) (1) Verification of diagnosis with checklist and ability to re-examine information. (2) Verification of diagnosis with checklist and without ability to re-examine information. (C) Pre-checklist diagnosis | Diagnostic accuracy on one of six clinical cases delivered by cardiopulmonary simulator | n=96 able to re-examine simulator, n=95 unable to re-examine simulator | Verification with a checklist improved diagnostic accuracy only for participants who could re-examine the simulator |
| Cognitive forcing strategies | ||||||
| Arzy, 200962 | Pretest–post-test | Attending doctors | (I/V) All groups instructed to diagnose cases, identifying one leading detail and rediagnose after removing it. (1) No warning of misleading detail. (2) Prior warning of misleading detail. (3) Trivial detail in place of misleading detail. (C) Baseline diagnosis | Diagnostic accuracy on 10 clinical scenarios | n=17 warning, n=17 no warning, n=17 trivial detail | Warning doctors to be cautious of misleading details had no effect on diagnostic accuracy; significant reduction in error resulted from re-examination following removal of leading detail |
| Feyzi-Behnagh, 201455 | Pretest–post-test | Residents | (I/V) (1) Instructions to diagnose case using consider alternatives scaffold; prompting participants to review their decision-making diagrams showing steps taken and correct steps. (2) Instructions to diagnose case using playback scaffold; showing participants the steps they took in reaching their diagnosis, followed by the correct steps. (C) Baseline diagnostic accuracy | Diagnostic accuracy on eight clinical scenarios, accuracy of confidence judgements | n=16 control, n=15 intervention | Both groups demonstrated learning gains from pretest to post-test; intervention group outperformed control group post-test in accuracy of confidence judgements |
| Regehr, 199456 | Within-subjects | First-year residents | (I/V) Instructions to argue for alternative diagnoses. (C) Instructions to diagnose cases based on first impression | Diagnostic accuracy on 12 slides of common dermatological diseases | n=32 completed control and intervention conditions | No statistically significant differences between intervention and control groups |
| Guided reflection | ||||||
| Mamede, 200839 | Within-subjects | Residents | (I/V) Instructions to interpret cases through guided reflective reasoning. (C) Instructions to interpret cases intuitively (‘the first diagnosis that comes to mind’) | Diagnostic accuracy on 16 clinical scenarios | n=42 completed control and intervention conditions | No main effect of reflective practice condition, but some improvement for complex cases |
| Mamede, 201042 | Within-subjects | Residents | (I/V) Instructions to interpret cases through guided reflective reasoning. (C) Instructions to interpret cases intuitively (‘the first diagnosis that comes to mind’) | Diagnostic accuracy on eight clinical scenarios with bias manipulation | n=36 completed control and intervention conditions | Reflective reasoning improves diagnostic accuracy over automatic reasoning in cases where bias manipulation is present |
| Mamede, 201263 | Within-subjects | Medical students, residents | (1) Conscious thought: instructions to diagnose case following elaborate analysis of information. (2) Unconscious thought: instructions to diagnose case after spending time-solving anagrams. (3) Instructions to diagnose cases based on first impression | Diagnostic accuracy on 12 clinical scenarios | n=84 completed all conditions | Reflective reasoning improves diagnostic accuracy over non-analytical reasoning for residents only, and is associated with fewer errors resulting from salient distracting clinical features |
| Myung, 201349 | Between-subjects | Fourth-year medical students | (I/V) Instructions to interpret cases through guided reflective reasoning. (C) No instructions | Diagnostic accuracy on four OSCE case vignettes | n=65 intervention, n=80 control | Intervention group significantly outperformed control group |
| Schmidt, 201458 | Within-subjects | Residents | (I/V) Instructions to interpret cases through guided reflective reasoning. (C) Instructions to diagnose case quickly and accurately after exposure to relevant information earlier in the day | Diagnostic accuracy on two clinical scenarios primed to trigger cognitive bias and two comparison cases | n=38 | Errors arising from cognitive bias triggered by prior exposure were reversed by guided reflection |
| Instructions at test | ||||||
| Ilgen, 201164 | Within-subjects | Senior medical students, junior and senior residents | (I/V) Instructions to diagnose cases using ‘directed search’ approach prompting structured analytical responses. (C) Instructions to diagnose cases based on first impression | Diagnostic accuracy on six simple and six complex clinical scenarios | n=115 completed control and intervention conditions | Across total sample, intervention group significantly outperformed control group |
| Ilgen, 201367 | Between-subjects | Senior medical students, residents, faculty clinicians | (I/V) Instructions to diagnose cases using ‘directed search’ approach prompting structured analytical responses. (C) Instructions to diagnose cases based on first impression | Diagnostic accuracy on 12 clinical scenarios | n=201 control, n=192 intervention | No statistically significant differences between intervention and control groups |
| Kulatunga-Moruzi, 200150 | Pretest–post-test | Medical students | (I/V) Instructions to diagnose case using clinical features list. (C) Instructions to diagnose cases based on first impression | Diagnostic accuracy on 12 slides of common dermatological diseases | n=16; randomised into two conditions | Control group outperformed intervention group when diagnosing similar cases across timepoints |
| Kulatunga-Moruzi, 201151 | Between-subjects | Medical students | (I/V) Instructions to diagnose case (1) based on first impressions, and then by listing all clinical features, (2) based on first impressions only, (3) by listing all clinical features only, (C) Instructions to diagnose case with no instructions | Diagnostic accuracy on 22 slides of common dermatological diseases | n=18 using clinical features, n=21 combined, n=12 no instruction | No significant differences between clinical features only and first impressions only. Dual instructions improved accuracy over no instructions, though did not reach statistical significance |
| Norman, 201457 | Between-subjects | Residents | (I/V) Instructions to diagnose reflectively and thoughtfully. (C) Instructions to diagnose quickly, without errors | Diagnostic accuracy on 20 case vignettes | n=108 intervention, n=96 control | No differences in accuracy; intervention condition took longer to reach diagnosis; incorrect answers took longer than correct answers |
| Sbbald, 201268 | Between-subjects | Fellows, residents, medical students | (I/V) Instructions to use dual-processing reasoning. (C) No instructions to use any specific reasoning approach | Diagnostic accuracy on eight cardiopulmonary simulator cases | n=26 experienced fellows (n=12 control), n=13 residents (n=6 control), n=23 medical students (n=12 control) | No overall effect of instruction |
| Sibbald, 201259 | Within-subjects | Residents | (I/V) Cases reinterpreted with instructions to approach diagnosis with (1) the same reasoning style as reported in control phase (potentiating) or (2) an alternative reasoning style to one reported in control phase (balancing). (C) Baseline free choice of reasoning style | Diagnostic accuracy on eight ECGs (each interpreted twice) | n=24 completed control and intervention conditions | No differences observed between potentiating and balancing instructions; improvement over initial diagnosis only with instructions to use analytical reasoning |
| Other interventions | ||||||
| Ibiapina, 201452 | Within-subjects | Senior medical students | (I/V) Instructions to diagnose cases using (1) modelled reflection or (2) cued reflection. (C) Instructions to diagnose case quickly and accurately | Diagnostic accuracy on eight clinical scenarios | n=39 modelled reflection group, n=39 cued reflection group, n=39 free reflection group | Modelled and cued reflection groups outperformed free reflection group, with no significant differences between modelled and cued groups |
| Mamede, 201065 | Within-subjects | Fourth-year medical students, residents | (I/V) (1) Conscious thought: instructions to diagnose case following elaborate analysis of information; or (2) unconscious thought: instructions to diagnose case after spending time-solving anagrams. (C) Instructions to diagnose cases based on first impression | Diagnostic accuracy on 12 clinical scenarios | n=84 completed all conditions | Conscious deliberation improves accuracy for complex cases for experts; no effect of experimental condition for simple cases. Novices benefit from deliberation without attention for simple cases |
| Payne, 201166 | Pretest–post-test | Medical students, residents | (I/V) Participants provided with feedback on diagnostic accuracy after entering final diagnosis for sample clinical scenarios in learning phase. (C) Participants provided with general disease information after entering final diagnosis for sample clinical scenarios in learning phase | Evidence of heuristic use from think-aloud protocol, eye-tracking; diagnostic accuracy on 15 clinical scenarios | n=20 intervention, n=20 control | No impact of metacognitive feedback intervention on use of heuristics or diagnostic accuracy |
C, control; DTI, diagnostic thinking inventory; ER, emergency room; I/V, intervention; OSCE, objective structured clinical examination.