Skip to main content
Log in

Do physicians know when their diagnoses are correct?

Implications for decision support and error reduction

  • Original Articles
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

Abstract

OBJECTIVE: This study explores the alignment between physicians’ confidence in their diagnoses and the “correctness” of these diagnoses, as a function of clinical experience, and whether subjects were prone to over-or underconfidence.

DESIGN: Prospective, counterbalanced experimental design.

SETTING: Laboratory study conducted under controlled conditions at three academic medical centers.

PARTICIPANTS: Seventy-two senior medical students, 72 senior medical residents, and 72 faculty internists.

INTERVENTION: We created highly detailed, 2-to 4-page synopses of 36 diagnostically challenging medical cases, each with a definitive correct diagnosis. Subjects generated a differential diagnosis for each of 9 assigned cases, and indicated their level of confidence in each diagnosis.

MEASUREMENTS AND MAIN RESULTS: A differential was considered “correct” if the clinically true diagnosis was listed in that subject’s hypothesis list. To assess confidence, subjects rated the likelihood that they would, at the time they generated the differential, seek assistance in reaching a diagnosis. Subjects’ confidence and correctness were “mildly” aligned (k=.314 for all subjects, .285 for faculty, .227 for residents, and .349 for students). Residents were overconfident in 41% of cases where their confidence and correctness were not aligned, whereas faculty were overconfident in 36% of such cases and students in 25%.

CONCLUSIONS: Even experienced clinicians may be unaware of the correctness of their diagnoses at the time they make them. Medical decision support systems, and other interventions designed to reduce medical errors, cannot rely exclusively on clinicians’ perceptions of their needs for such support.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Hersh WR. “A world of knowledge at your fingertips”: the promise, reality, and future directions of online information retrieval. Acad Med. 1999;74:240–3.

    Article  PubMed  CAS  Google Scholar 

  2. Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 2000.

    Google Scholar 

  3. Bates DW, Gawande AA. Error in medicine: what have we learned? Ann Intern Med. 2000;132:763–7.

    PubMed  CAS  Google Scholar 

  4. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995;274:35–43.

    Article  PubMed  CAS  Google Scholar 

  5. Wyatt JC. Clinical data systems, part 3: development and evaluation. Lancet. 1994;344:1682–7.

    Article  PubMed  CAS  Google Scholar 

  6. Norman DA. Melding mind and machine. Technol Rev. 1997;100:29–31.

    Google Scholar 

  7. Chueh H, Barnett GO. “Just in time” clinical information. Acad Med. 1997;72:512–7.

    Article  PubMed  CAS  Google Scholar 

  8. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes. JAMA. 1998;280:1339–46.

    Article  PubMed  CAS  Google Scholar 

  9. Miller RA. Medical diagnostic decision support systems—past, present, and future. J Am Med Inform Assoc. 1994;1:8–27.

    PubMed  CAS  Google Scholar 

  10. Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Eng J Med. 1998;338:232–8.

    Article  CAS  Google Scholar 

  11. McDonald CJ, Overhage JM, Tierney WM, et al. The Regenstrief Medical Record System: a quarter century experience. Int J Med Inform. 1999;54:225–53.

    Article  PubMed  CAS  Google Scholar 

  12. Wagner MM, Pankaskie M, Hogan W, et al. Clinical event monitoring at the University of Pittsburgh. Proc AMIA Annu Fall Symp. 1997;188–92.

  13. Cimino JJ, Elhanan G, Zeng Q. Supporting infobuttons with terminological knowledge. Proc AMIA Annu Fall Symp. 1997;528–32.

  14. Miller PL. Building an expert critiquing system: ESSENTIAL-ATTENDING. Methods Inf Med. 1986;25:71–8.

    PubMed  CAS  Google Scholar 

  15. Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science. 1974;185:1124–31.

    Article  Google Scholar 

  16. Lichtenstein S, Fischhoff B. Do those who know more also know more about how much they know? Organ Behav Hum Perform. 1977;20:159–83.

    Article  Google Scholar 

  17. Christensen-Szalanski JJ, Bushyhead JB. Physicians’ use of probabilistic information in a real clinical setting. J Exp Psychol. 1981;7:928–35.

    CAS  Google Scholar 

  18. Tierney WM, Fitzgerald J, McHenry R, et al. Physicians’ estimates of the probability of myocardial infarction in emergency room patients with chest pain. Med Decis Making. 1986;6:12–7.

    Article  PubMed  CAS  Google Scholar 

  19. Friedman CP, Elstein AS, Wolf FM, et al. Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: a multisite study of 2 systems. JAMA. 1999;282:1851–6.

    Article  PubMed  CAS  Google Scholar 

  20. Mann D. The relationship between diagnostic accuracy and confidence in medical students. Presented at the annual meeting of the American Educational Research Association, Atlanta, 1993.

  21. Friedman C, Gatti G, Elstein A, Franz T, Murphy G, Wolf F. Are Clinicians Correct When They Believe They Are Correct? Implications for Medical Decision Support. Proceedings of the Tenth World Congress on Medical Informatics. London; 2000. Medinfo. 2001;10(PP. 1):454–8.

    PubMed  CAS  Google Scholar 

  22. Swets JA, Pickett RM. Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. New York, NY: Academic Press; 1982.

    Google Scholar 

  23. McCullagh P, Nelder JA. Generalized Linear Models. 2nd ed. New York, NY: Chapman and Hall; 1991.

    Google Scholar 

  24. Liang K, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73:13–22.

    Article  Google Scholar 

  25. Neter J, Kutner MH, Nachstsheim CJ, Wasserman W. Applied Linear Regression Models. Chicago, IL: Irwin; 1996.

    Google Scholar 

  26. SAS Institute Inc. SAS/STAT User’s Guide, Version 8. Cary, NC: SAS Institute Inc.; 1999.

    Google Scholar 

  27. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

    Article  PubMed  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Charles P. Friedman PhD.

Additional information

The authors have no conflicts of interest to report.

This work was supported by grant R01-LM-05630 from the National Library of Medicine.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Friedman, C.P., Gatti, G.G., Franz, T.M. et al. Do physicians know when their diagnoses are correct?. J GEN INTERN MED 20, 334–339 (2005). https://doi.org/10.1111/j.1525-1497.2005.30145.x

Download citation

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1111/j.1525-1497.2005.30145.x

Key words

Navigation