Table 1

Characteristics of included studies

AuthorDesignParticipantsExperimental conditionsOutcome measureSample sizeFindings
Educational
 Hershberger, 199729Between-subjectsMedical students(I/V) Seminar teaching on major biases. (C) No seminarEvidence of heuristic use from performance on Inventory of Cognitive Bias in Medicine (ICBM), preintervention and postinterventionn=118 intervention, n=112 controlIntervention significantly improved performance on ICBM
 Nendaz, 201145Between-subjectsSixth-year medical students(I/V) Case-based reasoning seminars with emphasis on explicit reasoning steps, self-reflection and cognitive psychology concepts. (C) Standard case-based reasoning seminarsAccuracy of final and differential diagnoses on two assessment casesn=15 intervention, n=14 controlIntervention group performed better on differential diagnosis but not better on final accuracy of diagnosis
 Reilly, 201328Pretest–post-testResidents(I/V) Three-part, 1-year curriculum on cognitive bias and diagnostic error, including lectures, group-based discussion, online interactive modules and video vignettes. (C) No curriculumPerformance on the Diagnostic error Knowledge Assessment Test (13 items)n=38 intervention, n=42 controlKnowledge of cognitive biases and strategies significantly improved compared with baseline and control group
 Round, 199946Between-subjectsFourth-year medical students(I/V) Seminar consisting of small group discussion followed by teaching on cognitive biases and Bayes Theorem. (C) No seminarPerformance on Diagnostic Thinking Inventoryn=84 intervention, n=102 controlIntervention group significantly outperformed control group on DTI
 Schnapp, 201453Within-subjectsResidents(I/V) 30  min lecture on 10 key diagnostic errorsError identification on 10 clinical vignettes at 3-month follow-upn=37 completed preintervention and postinterventionSignificant improvement on only 3/10 error types
 Sherbino, 201447Between-subjectsSenior medical students(I/V) 90 min interactive seminar on cognitive forcing strategies. (C) No seminarEvidence of heuristic use from instigation of search for secondary diagnosis on six clinical scenariosn=145 intervention, n=46 controlImmediately after testing, <50% successfully show debiasing, vulnerable to false positives. No difference between intervention and control groups
Workplace
Checklist
 Graber, 201460Within-subjectsStaff-level ER physicians(I/V) Use of general and symptom-specific checklists on shiftExtensiveness of differential diagnosis on shifts, resource usagen=15 completed preintervention and postinterventionTrends towards more items in differential and ordering of more tests and consults with checklist use compared with baseline, though not statistically significant
 Shimizu, 201348Between-subjectsSenior medical students(1) Instructions to interpret (C) intuitively, then (I/V) using general debiasing checklist. (2) Instructions to interpret (C) intuitively, then (I/V) using differential diagnosis checklistDiagnostic accuracy on five clinical scenariosn=91 debiasing checklist, n=91 differential diagnosis checklistDifferential diagnosis checklist significantly improves diagnostic accuracy over intuitive reasoning; no effect of general debiasing checklist
 Sibbald, 201361Within-subjectsCardiology fellows(1) Verification with checklist. (2) Interpretation and verification with checklist. (3) Undirected interpretation. (4) Interpretation and verification without checklist.Diagnostic accuracy on 18 ECGsn=15 completed all conditionsChecklist use resulted in more correction of errors in verification conditions and fewer errors overall
 Sibbald, 201354Pretest–post-testResidents(I/V) (1) Verification of diagnosis with checklist and ability to re-examine information. (2) Verification of diagnosis with checklist and without ability to re-examine information. (C) Pre-checklist diagnosisDiagnostic accuracy on one of six clinical cases delivered by cardiopulmonary simulatorn=96 able to re-examine simulator, n=95 unable to re-examine simulatorVerification with a checklist improved diagnostic accuracy only for participants who could re-examine the simulator
Cognitive forcing strategies
 Arzy, 200962Pretest–post-testAttending doctors(I/V) All groups instructed to diagnose cases, identifying one leading detail and rediagnose after removing it. (1) No warning of misleading detail. (2) Prior warning of misleading detail. (3) Trivial detail in place of misleading detail. (C) Baseline diagnosisDiagnostic accuracy on 10 clinical scenariosn=17 warning, n=17 no warning, n=17 trivial detailWarning doctors to be cautious of misleading details had no effect on diagnostic accuracy; significant reduction in error resulted from re-examination following removal of leading detail
 Feyzi-Behnagh, 201455Pretest–post-testResidents(I/V) (1) Instructions to diagnose case using consider alternatives scaffold; prompting participants to review their decision-making diagrams showing steps taken and correct steps. (2) Instructions to diagnose case using playback scaffold; showing participants the steps they took in reaching their diagnosis, followed by the correct steps. (C) Baseline diagnostic accuracyDiagnostic accuracy on eight clinical scenarios, accuracy of confidence judgementsn=16 control, n=15 interventionBoth groups demonstrated learning gains from pretest to post-test; intervention group outperformed control group post-test in accuracy of confidence judgements
 Regehr, 199456Within-subjectsFirst-year residents(I/V) Instructions to argue for alternative diagnoses. (C) Instructions to diagnose cases based on first impressionDiagnostic accuracy on 12 slides of common dermatological diseasesn=32 completed control and intervention conditionsNo statistically significant differences between intervention and control groups
Guided reflection
 Mamede, 200839Within-subjectsResidents(I/V) Instructions to interpret cases through guided reflective reasoning. (C) Instructions to interpret cases intuitively (‘the first diagnosis that comes to mind’)Diagnostic accuracy on 16 clinical scenariosn=42 completed control and intervention conditionsNo main effect of reflective practice condition, but some improvement for complex cases
 Mamede, 201042Within-subjectsResidents(I/V) Instructions to interpret cases through guided reflective reasoning. (C) Instructions to interpret cases intuitively (‘the first diagnosis that comes to mind’)Diagnostic accuracy on eight clinical scenarios with bias manipulationn=36 completed control and intervention conditionsReflective reasoning improves diagnostic accuracy over automatic reasoning in cases where bias manipulation is present
 Mamede, 201263Within-subjectsMedical students, residents (1) Conscious thought: instructions to diagnose case following elaborate analysis of information. (2) Unconscious thought: instructions to diagnose case after spending time-solving anagrams. (3) Instructions to diagnose cases based on first impressionDiagnostic accuracy on 12 clinical scenariosn=84 completed all conditionsReflective reasoning improves diagnostic accuracy over non-analytical reasoning for residents only, and is associated with fewer errors resulting from salient distracting clinical features
 Myung, 201349Between-subjectsFourth-year medical students(I/V) Instructions to interpret cases through guided reflective reasoning. (C) No instructionsDiagnostic accuracy on four OSCE case vignettesn=65 intervention, n=80 controlIntervention group significantly outperformed control group
 Schmidt, 201458Within-subjectsResidents(I/V) Instructions to interpret cases through guided reflective reasoning. (C) Instructions to diagnose case quickly and accurately after exposure to relevant information earlier in the dayDiagnostic accuracy on two clinical scenarios primed to trigger cognitive bias and two comparison casesn=38Errors arising from cognitive bias triggered by prior exposure were reversed by guided reflection
Instructions at test
 Ilgen, 201164Within-subjectsSenior medical students, junior and senior residents(I/V) Instructions to diagnose cases using ‘directed search’ approach prompting structured analytical responses. (C) Instructions to diagnose cases based on first impressionDiagnostic accuracy on six simple and six complex clinical scenariosn=115 completed control and intervention conditionsAcross total sample, intervention group significantly outperformed control group
 Ilgen, 201367Between-subjectsSenior medical students, residents, faculty clinicians(I/V) Instructions to diagnose cases using ‘directed search’ approach prompting structured analytical responses. (C) Instructions to diagnose cases based on first impressionDiagnostic accuracy on 12 clinical scenariosn=201 control, n=192 interventionNo statistically significant differences between intervention and control groups
 Kulatunga-Moruzi, 200150Pretest–post-testMedical students(I/V) Instructions to diagnose case using clinical features list. (C) Instructions to diagnose cases based on first impressionDiagnostic accuracy on 12 slides of common dermatological diseasesn=16; randomised into two conditionsControl group outperformed intervention group when diagnosing similar cases across timepoints
 Kulatunga-Moruzi, 201151Between-subjectsMedical students(I/V) Instructions to diagnose case (1) based on first impressions, and then by listing all clinical features, (2) based on first impressions only, (3) by listing all clinical features only, (C) Instructions to diagnose case with no instructionsDiagnostic accuracy on 22 slides of common dermatological diseasesn=18 using clinical features, n=21 combined, n=12 no instructionNo significant differences between clinical features only and first impressions only. Dual instructions improved accuracy over no instructions, though did not reach statistical significance
 Norman, 201457Between-subjectsResidents(I/V) Instructions to diagnose reflectively and thoughtfully. (C) Instructions to diagnose quickly, without errorsDiagnostic accuracy on 20 case vignettesn=108 intervention, n=96 controlNo differences in accuracy; intervention condition took longer to reach diagnosis; incorrect answers took longer than correct answers
 Sbbald, 201268Between-subjectsFellows, residents, medical students(I/V) Instructions to use dual-processing reasoning. (C) No instructions to use any specific reasoning approachDiagnostic accuracy on eight cardiopulmonary simulator casesn=26 experienced fellows (n=12 control), n=13 residents (n=6 control), n=23 medical students (n=12 control)No overall effect of instruction
 Sibbald, 201259Within-subjectsResidents(I/V) Cases reinterpreted with instructions to approach diagnosis with (1) the same reasoning style as reported in control phase (potentiating) or (2) an alternative reasoning style to one reported in control phase (balancing). (C) Baseline free choice of reasoning styleDiagnostic accuracy on eight ECGs (each interpreted twice)n=24 completed control and intervention conditionsNo differences observed between potentiating and balancing instructions; improvement over initial diagnosis only with instructions to use analytical reasoning
Other interventions
 Ibiapina, 201452Within-subjectsSenior medical students(I/V) Instructions to diagnose cases using (1) modelled reflection or (2) cued reflection. (C) Instructions to diagnose case quickly and accuratelyDiagnostic accuracy on eight clinical scenariosn=39 modelled reflection group, n=39 cued reflection group, n=39 free reflection groupModelled and cued reflection groups outperformed free reflection group, with no significant differences between modelled and cued groups
 Mamede, 201065Within-subjectsFourth-year medical students, residents(I/V) (1) Conscious thought: instructions to diagnose case following elaborate analysis of information; or (2) unconscious thought: instructions to diagnose case after spending time-solving anagrams. (C) Instructions to diagnose cases based on first impressionDiagnostic accuracy on 12 clinical scenariosn=84 completed all conditionsConscious deliberation improves accuracy for complex cases for experts; no effect of experimental condition for simple cases. Novices benefit from deliberation without attention for simple cases
 Payne, 201166Pretest–post-testMedical students, residents(I/V) Participants provided with feedback on diagnostic accuracy after entering final diagnosis for sample clinical scenarios in learning phase. (C) Participants provided with general disease information after entering final diagnosis for sample clinical scenarios in learning phaseEvidence of heuristic use from think-aloud protocol, eye-tracking; diagnostic accuracy on 15 clinical scenariosn=20 intervention, n=20 controlNo impact of metacognitive feedback intervention on use of heuristics or diagnostic accuracy
  • C, control; DTI, diagnostic thinking inventory; ER, emergency room; I/V, intervention; OSCE, objective structured clinical examination.