You are here

  • Home
  • Archive
  • Volume 23, Issue 9
  • The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations

Article Text

Article menu
Original research
The frequency of diagnostic errors in outpatient care: estimations from three large observational studies involving US adult populations
  1. Hardeep Singh1,
  2. Ashley N D Meyer1,
  3. Eric J Thomas2
  1. 1Houston Veterans Affairs Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center and the Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
  2. 2Division of General Medicine, Department of Medicine, University of Texas at Houston, Memorial Hermann Center for Healthcare Quality and Safety, University of Texas Medical School at Houston, Houston, Texas, USA
  1. Correspondence to Dr Hardeep Singh, VA Medical Center (152), 2002 Holcombe Blvd, VAMC 152, Houston, TX 77030, USA; hardeeps{at}bcm.edu

Abstract

Background The frequency of outpatient diagnostic errors is challenging to determine due to varying error definitions and the need to review data across multiple providers and care settings over time. We estimated the frequency of diagnostic errors in the US adult population by synthesising data from three previous studies of clinic-based populations that used conceptually similar definitions of diagnostic error.

Methods Data sources included two previous studies that used electronic triggers, or algorithms, to detect unusual patterns of return visits after an initial primary care visit or lack of follow-up of abnormal clinical findings related to colorectal cancer, both suggestive of diagnostic errors. A third study examined consecutive cases of lung cancer. In all three studies, diagnostic errors were confirmed through chart review and defined as missed opportunities to make a timely or correct diagnosis based on available evidence. We extrapolated the frequency of diagnostic error obtained from our studies to the US adult population, using the primary care study to estimate rates of diagnostic error for acute conditions (and exacerbations of existing conditions) and the two cancer studies to conservatively estimate rates of missed diagnosis of colorectal and lung cancer (as proxies for other serious chronic conditions).

Results Combining estimates from the three studies yielded a rate of outpatient diagnostic errors of 5.08%, or approximately 12 million US adults every year. Based upon previous work, we estimate that about half of these errors could potentially be harmful.

Conclusions Our population-based estimate suggests that diagnostic errors affect at least 1 in 20 US adults. This foundational evidence should encourage policymakers, healthcare organisations and researchers to start measuring and reducing diagnostic errors.

  • Trigger Tools
  • Diagnostic Errors
  • Patient Safety
  • Chart Review Methodologies
  • Medical Error, Measurement/Epidemiology

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

https://doi.org/10.1136/bmjqs-2013-002627

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Background

    Diagnostic errors pose an important threat to healthcare quality and safety in outpatient settings.1 ,2 The frequency of outpatient diagnostic errors is unknown,3 due in part to varying definitions across studies and to the challenge of detecting these errors, which typically emerge across multiple episodes of care over time. A recent review by Graber discussed the use of several methods to study diagnostic errors, including autopsies, case reviews, surveys, incident reporting, standardised patients, second reviews and malpractice claims.3 However, most of these methods have limited use in determining the frequency of diagnostic error in outpatient medical practice.

    Population-based estimates of diagnostic error could help prioritise ongoing patient safety efforts. We aimed to estimate the frequency of outpatient diagnostic errors in the US adult population by synthesising data from three previous studies of clinic-based populations that used conceptually similar definitions of diagnostic error.4–6

    Methods

    Data sources included two previous studies that used electronic triggers, or algorithms, to detect unusual patterns of return visits (primary care study) or lack of follow-up of abnormal clinical findings for colorectal cancer (CRC) (colon cancer study), both suggestive of diagnostic errors.4 ,6 A third study examined consecutive cases of lung cancer in two institutions (lung cancer study).5 In all three studies, diagnostic errors were confirmed through chart review and defined as missed opportunities to make a timely or correct diagnosis based on available evidence. The criteria for diagnostic errors were comparable across the three studies and excluded atypical presentations and appropriate decisions to watch and wait.

    The triggers in the primary care study6 were based on the occurrence of unexpected return visits and were therefore selective for misdiagnosis of acute conditions and exacerbations of chronic conditions. Trigger 1 was a primary care index visit followed by unplanned hospitalisation within 14 days; trigger 2 was a primary care index visit followed by one or more unscheduled visit(s) within 14 days. Control visits did not meet either criterion. We considered for inclusion all primary care visits (n=212 165 visits and 81 483 unique patients) in two large health systems between 1 October 2006 and 30 September 2007. For the record review, we reviewed all trigger 1 records from one of the sites (n=220 records) and randomly chose the rest from the second site, yielding a reviewed sample of 674 records out of the total of 1086 visits selected. Maintaining a record sampling ratio from each health system that was consistent with trigger 1, we reviewed 669 of 14 777 visits selected by trigger 2 and 614 of 193 810 control non-triggered visits. Errors in triggered and non-triggered (control) cases were judged to have occurred when adequate data to suggest the final, correct diagnosis existed at an earlier primary care visit, or if documented abnormal findings at the visit should have prompted additional evaluation that would have revealed the correct diagnosis. Determination of errors was made on retrospective review by two independent raters. In extrapolating error rates to all primary care patients at our study sites, we assumed that rates of error in randomly selected samples of triggered and non-triggered cases were applicable to the entire cohort of triggered and non-triggered records in each clinic population.

    Although we presumed that the types of errors detected in the primary care study were likely to account for the majority of outpatient diagnostic errors, we also aimed to estimate the rate of errors related to initial diagnosis of less common but serious chronic conditions. Unlike misdiagnosis of acute conditions, these errors may not become apparent for weeks or months. As a proxy for such errors in outpatient care, we obtained frequency estimates for missed and delayed diagnosis of CRC and lung cancer, represented in the two cancer studies,4 ,5 respectively. These conditions were selected due to their relative frequency in several types of prior observational studies related to diagnostic errors.5 ,7–12 In both of our studies, diagnostic errors were characterised by lack of timely follow-up on specific ‘red flags’ or ‘alarm’ features5 ,10 ,13 of cancer in patients who were eventually diagnosed with either cancer. For instance, for CRC this was operationally defined as lack of follow-up 60 days after documentation of hematochezia (bright red blood per rectum), positive faecal occult blood test and iron deficiency anaemia. For lung cancer, we defined lack of follow-up as failure to initiate further investigation or an appropriate action within 7 days of a documented ‘red flag’ such as an abnormal chest X-ray. None of these data overlapped with the primary care study. We assumed that failure to follow-up on these ‘red flags’ accounted for all diagnostic errors associated with these types of cancer.

    We extrapolated the frequency of diagnostic error obtained from our studies to the US adult population,14 using the primary care study as a proxy to estimate rates of diagnostic error for acute conditions (and exacerbations of existing conditions) and the two cancer studies to conservatively estimate rates of missed diagnosis of CRC and lung cancer (as proxies for other serious chronic conditions). Because of the heterogeneity of the conditions we included, we assumed that they represented the spectrum of diseases at risk for outpatient diagnostic errors.

    Results

    In the primary care study, samples of triggered visits and control visits were reviewed to determine the percentages of visits that contained errors. These percentages were extrapolated to the larger triggered and non-triggered samples. For trigger 1, 141 errors were found in 674 visits reviewed, yielding an error rate of 20.9%. Extrapolating to all 1086 trigger 1 visits yielded an estimate of 227.2 errors. For trigger 2, 36 errors were found in 669 visits reviewed, yielding an error rate of 5.4%. Extrapolating to all 14 777 trigger 2 visits yielded an estimate of 795.2 errors. Finally, for the control visits, 13 errors were found in 614 visits reviewed, yielding an error rate of 2.1%. Extrapolating to all 193 810 control visits yielded an estimate of 4103.5 errors. Thus, we estimated that 5126 errors would have occurred across the three groups. We then divided this figure by the number of unique primary care patients in the initial cohort (81 483) and arrived at an estimated error rate of 6.29%. Because approximately 80.5% of US adults seek outpatient care annually,15 the same rate when applied to all US adults gives an estimate of 5.06% (see figure 1).

    Figure 1
    Figure 1

    Schematic of synthesis of diagnostic error frequency data from three studies.

    The colon cancer study estimated errors in CRC diagnosis in 26 out of 291 773 patients seen in outpatient care (0.009%). Extrapolating to the US adult population that seeks outpatient care, we estimated 0.007% of all adults would have errors related to CRC per year. In the lung cancer study, we found 127 errors in 587 cases of lung cancer, yielding a 21.64% error rate. Extrapolating to the 80.5% of the US adult population that seeks outpatient care and the 0.072% of the population that becomes newly diagnosed with lung cancer each year,16 we estimate 0.013% of all adults would have errors related to lung cancer per year.

    Combining estimates from the three studies yields a rate of outpatient diagnostic errors of 5.08%, or approximately 12 million US adults every year. Based on our previous work,17 we estimated that about one-half of errors would have the potential to lead to severe harm.

    Discussion

    Although diagnostic errors are difficult to define and measure, there is mounting evidence of their importance and frequency across several types of conditions.3 ,7 ,11 ,18 Applying aggregate diagnostic error rates from three previous studies conducted by our group, we estimate that approximately 12 million adults in the USA could experience outpatient diagnostic errors each year. Our estimates are generally consistent with data from the general public about diagnostic errors. A recent telephone survey of a random probability sample of over 2000 US adults suggested that more than one in 10 respondents reported a ‘diagnostic mishap’.19 Similarly, a survey of 726 paediatricians revealed that more than half (54%) reported making a diagnostic error at least once or twice per month.20 Thus, other recent data corroborate our finding that diagnostic errors are common.

    Previous estimations of the frequency of diagnostic error largely relied on expert opinion. For example, Elstein estimated errors occur in about 10–15% of diagnoses.21 As noted by Graber, many methods to study diagnostic errors have limited use in calculating the frequency of the problem. For example, autopsy rates have declined,22 the number of self-reports remains low and malpractice claims data are prone to bias and are non-representative.11 ,20 ,23–25 On the other hand, population-based studies have relied on medical record reviews, which are generally a gold standard for determining diagnostic error.26 Our electronic trigger approach facilitated selective record reviews, thus making the error determination process more efficient than either consecutive or random record reviews. Our estimate, based on a large sample size, is likely to be the most robust one thus far to address the frequency of diagnostic error in routine outpatient practice.

    Although it is unknown how many patients will be harmed from diagnostic errors, our previous work17 suggests that about one-half of diagnostic errors have the potential to lead to severe harm. While this is only an estimate and does not imply all those affected will actually have harm, this risk potentially translates to about 6 million outpatients per year. Additionally, while the contribution of the two cancers to the overall estimation was small, we believed it was important to include them to show their relative contribution. This is because delayed cancer diagnosis is believed to be one of the most harmful and costly types of diagnostic error in the outpatient setting and its significance has become apparent not only in malpractice claims but also in retrospective studies of consecutive cancer cases, surveys and studies of failures to follow-up abnormal test results.5 ,7–12 In many of these studies, lung cancer and CRC are the most common cancers for which diagnosis is delayed.

    Our estimates are limited by our assumptions of generalisability. To keep the definition of diagnostic error uniform across studies, we only used data from our own work. Other studies have used different methods and definitions of diagnostic errors and we believed that this would make our extrapolations weaker. Similar to other methods to study medical errors, our trigger was not sensitive enough to capture all types of diagnostic errors in outpatient settings. Our triggers did not adequately capture errors in diagnosis of uncommon diseases, some of which might evolve over a prolonged period of time in the outpatient setting. Our studies also might not have adequately addressed other types of common chronic conditions that could be vulnerable to diagnostic error.27 For example, a recent study from the UK suggests that opportunities to diagnose chronic obstructive pulmonary disease at an earlier stage are being missed in the majority of cases.28 While it is tempting to do so, we did not extrapolate our chronic condition proxy estimates to all types of cancer or other types of chronic diseases because it would not be possible to do this accurately. Lastly, chart reviews inherently would miss errors due to documentation-related issues and this false negative rate is largely unknown. All of these limitations thus suggest that we underestimated the frequency of diagnostic errors and frequency could be well over 5%.

    In conclusion, we estimate the frequency of diagnostic error to be at least 5% in US outpatient adults, a number that entails a substantial patient safety risk. This population-based estimate should provide a foundation for policymakers, healthcare organisations and researchers to strengthen efforts to measure and reduce diagnostic errors.

    Acknowledgments

    We thank Annie Bradford, PhD for assistance with medical editing.

    References

    1. ↵
      1. Singh H,
      2. Thomas E,
      3. Khan MM,
      4. et al
      . Identifying diagnostic errors in primary care using an electronic screening algorithm. Arch Intern Med 2007;167:302–8.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Singh H,
      2. Weingart SN
      . Diagnostic errors in ambulatory care: dimensions and preventive strategies. Adv Health Sci Educ Theory Pract 2009;14(Suppl 1):57–61.
      OpenUrlCrossRefPubMed
    3. ↵
      1. Graber ML
      . The incidence of diagnostic error in medicine. BMJ Qual Saf 2013;22(Suppl 2):ii21–7.
      OpenUrlAbstract/FREE Full Text
    4. ↵
      1. Murphy DR,
      2. Laxmisan A,
      3. Reis BA,
      4. et al
      . Electronic health record-based triggers to detect potential delays in cancer diagnosis. BMJ Qual Saf 2014;23:8–16.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Singh H,
      2. Hirani K,
      3. Kadiyala H,
      4. et al
      . Characteristics and predictors of missed opportunities in lung cancer diagnosis: an electronic health record-based study. J Clin Oncol 2010;28:3307–15.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Singh H,
      2. Giardina TD,
      3. Forjuoh SN,
      4. et al
      . Electronic health record-based surveillance of diagnostic errors in primary care. BMJ Qual Saf 2012;21:93–100.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Schiff GD,
      2. Hasan O,
      3. Kim S,
      4. et al
      . Diagnostic error in medicine: analysis of 583 physician-reported errors. Arch Intern Med 2009;169:1881–7.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Singh H,
      2. Kadiyala H,
      3. Bhagwath G,
      4. et al
      . Using a multifaceted approach to improve the follow-up of positive fecal occult blood test results. Am J Gastroenterol 2009;104:942–52.
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Singh H,
      2. Thomas EJ,
      3. Mani S,
      4. et al
      . Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential? Arch Intern Med 2009;169:1578–86.
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Singh H,
      2. Daci K,
      3. Petersen L,
      4. et al
      . Missed opportunities to initiate endoscopic evaluation for colorectal cancer diagnosis. Am J Gastroenterol 2009;104:2543–54.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Gandhi TK,
      2. Kachalia A,
      3. Thomas EJ,
      4. et al
      . Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Ann Intern Med 2006;145:488–96.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Schiff GD,
      2. Puopolo AL,
      3. Huben-Kearney A,
      4. et al
      . Primary care closed claims experience of Massachusetts malpractice insurers. JAMA Intern Med 2013;173:2063–8.
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      Jones R, Latinovic R, Charlton J, et al. Alarm symptoms in early diagnosis of cancer in primary care: cohort study using General Practice Research Database. BMJ 2007;334:1040.
    14. ↵
      Annual Estimates of the Resident Population by Sex, Age, Race, and Hispanic Origin for the United States and States: 1 April 2010 to 1 July 2012. US Census Bureau, Population Division [serial online] 2013; (accessed 18 Feb 2014). http://factfinder2.census.gov/bkmk/table/1.0/en/PEP/2012/PEPASR6H.
    15. ↵
      1. Schiller JS,
      2. Lucas JW,
      3. Ward BW,
      4. et al
      . Summary health statistics for U.S. adults: National Health Interview Survey, 2010. Vital Health Stat 2012;10:1–207.
    16. ↵
      American Cancer Society. Cancer facts & figures. Atlanta: American Cancer Society, 2012. Ref Type: Report
    17. ↵
      1. Singh H,
      2. Giardina TD,
      3. Meyer AN,
      4. et al
      . Types and origins of diagnostic errors in primary care settings. JAMA Intern Med 2013:1–8.
    18. ↵
      1. Weingart SN,
      2. Saadeh MG,
      3. Simchowitz B,
      4. et al
      . Process of care failures in breast cancer diagnosis. J Gen Intern Med 2009;24:702–9.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      Patient Experience March 2009. Consumer Reports National Research Center [serial online] 2013; (accessed 28 Oct 2013). http://consumersunion.org/pdf/medical-error-poll-309.pdf
    20. ↵
      1. Singh H,
      2. Thomas EJ,
      3. Wilson L,
      4. et al
      . Errors of diagnosis in pediatric practice: a multisite survey. Pediatrics 2010;126:70–9.
      OpenUrlAbstract/FREE Full Text
    21. ↵
      1. Elstein AS
      . Clinical reasoning in medicine. In: Higgs J, Jones MA. eds Clinical reasoning in the health professions. Woburn, MA: Butterworth-Heinemann, 1995:49–59.
    22. ↵
      1. Shojania KG,
      2. Burton EC
      . The vanishing nonforensic autopsy. N Engl J Med 2009;358:873–5.
      OpenUrlWeb of Science
    23. ↵
      1. Thomas EJ,
      2. Petersen LA
      . Measuring errors and adverse events in health care. J Gen Intern Med 2003;18:61–7.
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. McAbee GN,
      2. Donn SM,
      3. Mendelson RA,
      4. et al
      . Medical diagnoses commonly associated with pediatric malpractice lawsuits in the United States. Pediatrics 2008;122:e1282–6.
      OpenUrlAbstract/FREE Full Text
    25. ↵
      1. Phillips R Jr.,
      2. Bartholomew LA,
      3. Dovey SM,
      4. et al
      . Learning from malpractice claims about negligent, adverse events in primary care in the United States. Qual Saf Health Care 2004;13:121–6.
      OpenUrlAbstract/FREE Full Text
    26. ↵
      1. Thomas EJ,
      2. Studdert DM,
      3. Burstin HR,
      4. et al
      . Incidence and types of adverse events and negligent care in Utah and Colorado in 1992. Med Care 2000;38:261–71.
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Kostopoulou O,
      2. Delaney BC,
      3. Munro CW
      . Diagnostic difficulty and error in primary care—a systematic review. Fam Pract 2008;25:400–13.
      OpenUrlAbstract/FREE Full Text
    28. ↵
      1. Jones RCM,
      2. Price D,
      3. Ryan D,
      4. et al
      . Opportunities to diagnose chronic obstructive pulmonary disease in routine care in the UK: a retrospective study of a clinical cohort. Lancet Respir Med 2014; Early Online Publication. 13 Feb 2014. doi:10.1016/S2213-2600(14)70008-6H

    Supplementary materials

    Footnotes

    • Contributors All authors made substantial contributions to conception and design, and/or acquisition of data, and/or analysis and interpretation of data; all authors participated in drafting the article or revising it critically for important intellectual content; all authors gave final approval of the version published; and all authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

    • Funding Studies mentioned in this paper were supported by an NIH K23 Career Development Award (K23CA125585), the VA National Center of Patient Safety, and Agency for Healthcare Research and Quality (R18HS017820 and R18HS17244-02). HS and ANDM were supported in part by the Houston VA Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413).

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.