You are here

Article Text

Article menu

Download PDFPDF

Original research
Challenges of making a diagnosis in the outpatient setting: a multi-site survey of primary care physicians
  1. Urmimala Sarkar1,2,
  2. Doug Bonacum3,
  3. William Strull4,
  4. Christiane Spitzmueller5,
  5. Nancy Jin1,2,
  6. Andrea López1,2,
  7. Traber Davis Giardina6,
  8. Ashley N D Meyer6,
  9. Hardeep Singh6
  1. 1Division of General Internal Medicine, University of California, San Francisco, San Francisco, California, USA
  2. 2Center for Vulnerable Populations, San Francisco General Hospital, University of California, San Francisco, San Francisco, California, USA
  3. 3Kaiser Permanente, Oakland, California, USA
  4. 4The Permanente Federation at Kaiser-Permanente, Oakland, California, USA
  5. 5University of Houston, Houston, Texas, USA
  6. 6Houston VA HSR&D Center of Excellence, Michael E. DeBakey Veterans Affairs Medical Center and the Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Texas, USA
  1. Correspondence to Dr Urmimala Sarkar, Department of Medicine, SFGH Division of General Internal Medicine, University of California, San Francisco, Box 1364, 1001 Potrero, Bldg 10, Floor 3, Ward 13, San Francisco, CA 94143-1364, USA; usarkar{at}medsfgh.ucsf.edu

Abstract

Background Although misdiagnosis in the outpatient setting leads to significant patient harm and wasted resources, it is not well studied. The authors surveyed primary care physicians (PCPs) about barriers to timely diagnosis in the outpatient setting and assessed their perceptions of diagnostic difficulty.

Methods Surveys of PCPs practicing in an integrated health system across 10 geographically dispersed states in 2005. The survey elicited information on key cognitive failures (including in clinical knowledge or judgement) for a specific case, and solicited strategies for reducing diagnostic delays. Content analysis was used to categorise cognitive failures and strategies for improvement. The authors examined the extent and predictors of diagnostic difficulty, defined as reporting >5% patients difficult to diagnose.

Results Of 1817 physicians surveyed, 1054 (58%) responded; 848 (80%) respondents primarily practiced in outpatient settings and had an assigned patient panel (inclusion sample). Inadequate knowledge (19.9%) was the most commonly reported cognitive factor. Half reported >5% of their patients were difficult to diagnose; more experienced physicians reported less diagnostic difficulty. In adjusted analyses, problems with information processing (information availability and time to review it) and the referral process were associated with greater diagnostic difficulty. Strategies for improvement most commonly involved workload issues (panel size, non-visit tasks).

Conclusions PCPs report a variety of reasons for diagnostic difficulties in primary care practice. In this study, knowledge gaps appear to be a prominent concern. Interventions that address these gaps as well as practice level issues such as time to process diagnostic information and better subspecialty input may reduce diagnostic difficulties in primary care.

  • Diagnostic error
  • missed and delayed diagnosis
  • primary care
  • cognitive errors
  • patient safety
  • information technology
  • trigger tools
  • health services research

https://doi.org/10.1136/bmjqs-2011-000541

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Missed, delayed and incorrect diagnoses (also commonly labelled as misdiagnosis) lead to significant patient harm and wasted medical resources but remain understudied, especially in the ambulatory setting.1–6 The diagnostic process is complex and involves multiple providers and settings over a period of time, and thus comprehensive investigation of this issue remains elusive. Prior work examining closed outpatient malpractice claims suggests that failures of clinical judgement, process breakdowns, and patient factors commonly contributed to missed and delayed diagnoses.7 ,8 However, claims-based studies are not always generalisable to primary care practice.9

    Physician perspectives are essential in enhancing understanding of missed and delayed diagnosis. For instance, a recent physician survey solicited specific cases of diagnostic errors from internists revealing that, across multiple diseases, process breakdowns in the testing phase were commonly implicated.4 In another study on outpatient safety, primary care physicians (PCPs) believed that poor processes of care involving test results contributed significantly to medical errors.10 While these physician perspectives provide useful information about the types of missed and delayed diagnosis and establish the relationship of errors with process breakdowns, workflow and process issues that lead to diagnostic difficulty are not well understood. For instance, difficulty in achieving timely and correct diagnosis might relate to factors that are disease-specific, such as atypical disease presentation,11 but may also result from factors related to availability and interpretation of diagnostic information, support from specialty consultants, and the PCP's work environment.

    Physicians are often unaware of the diagnostic errors they make and lack systematic feedback systems to provide them with information about which of their patients experienced missed or delayed diagnosis.12 However, they can readily report difficulties they experience in the diagnostic processes,10 and this knowledge could further advance the understanding of missed and delayed diagnosis. Therefore, our objective was to survey PCPs practicing in an integrated health system across multiple US regions about potential barriers to timely diagnosis in the outpatient setting and the diagnostic difficulty they faced in their practices. By reflecting on their own specific case of missed or delayed diagnosis, physicians reported key cognitive failures (such as in clinical knowledge or judgement) that occurred in the case. We also solicited their opinions on strategies to reduce diagnostic delays.

    Methods

    This survey was conducted as part of an internal quality improvement process within a large, integrated healthcare system. The goal of this programme was to characterise the vulnerabilities in the outpatient diagnostic process. In addition to the survey discussed in this report, the project included focus groups of physicians and, separately, health plan member patients, discussing the diagnostic process. The information was used by the organisational leadership to attempt to improve the outpatient diagnostic process.

    Study population

    The target population included salaried, adult PCPs (family medicine or internal medicine) practicing within five distinct geographic regions of a large integrated health system. The survey was done as part of an internal quality improvement initiative in 2005. The research team identified potential participants through credentialing departments, continuing medical education departments and published physician lists. Subspecialists were not included.

    Survey development and outcome measures

    Because validated measures about diagnostic difficulty and barriers to timely diagnosis are underdeveloped, we developed questions based on a comprehensive literature review and analyses of medical–legal claims. The survey instrument was pilot-tested through cognitive interviews with 15 physicians and modified accordingly.13–15

    To determine the extent of perceived diagnostic difficulty, we used the item, ‘In the past year, about what percentage of your patients did you consider difficult to diagnose?’ with five ordered responses of 0%, 1%–5%, 6%–10%, 11%–15% and >15%. This was our primary outcome of interest.

    Respondents reported age, gender, region and year in practice as well as years in the health system. Because we were mostly interested in PCP perspectives, we obtained information on time practiced in the outpatient versus other (hospital, urgent care, nursing home) settings and whether participants managed an outpatient panel. The survey evaluated potential barriers to timely diagnosis related to three conceptual domains: (1) processing of diagnostic information such as from symptoms, signs and tests, (2) referral processes and (3) patient characteristics. We selected these domains because of their association with the diagnostic process and because several aspects of these domains were measurable using a PCP-focused structured survey instrument.

    The survey also included two open-ended items that: (1) elicited key failures in clinical knowledge or judgement that occurred in a specific case of delayed diagnosis recalled by the participant and (2) requested suggestions to reduce delays in diagnosis, respectively. For the knowledge and judgement failures, the research team (with expertise in cognitive psychology and diagnostic error) used an established taxonomy of cognitive errors5 ,16 and used content analysis to categorise the described failures into one or more cognitive errors. Some respondents did not provide sufficient information for categorisation and their responses were excluded. For the suggestions to reduce delays, a trained qualitative analyst used content analysis with open coding to describe the free-text responses, and the research team categorised them into several categories of interventions targeting systems and process issues to improve the diagnostic process.

    Survey administration

    An independent research firm administered the final 54-item confidential survey via mail to participants in March 2005. Physicians were offered a $10 incentive to participate, and non-responders received mailed reminders 7 and 14 days after the survey was distributed. The research firm performed all of the data collection and entry and de-identified results for analysis.

    Measure testing

    We performed an explanatory factor analysis to ensure a single underlying construct for each scale related to the three conceptual domains (processing of diagnostic information, referral processes and patient characteristics). We calculated Cronbach's α for each measure to determine reliability levels. Based on the results, items mapped to three underlying factors, corresponding to three domains, as follows: (1) information processing, including availability of information and time to review it (five items, range 1–5, one factor, eigenvalue=1.8, Cronbach α=0.70); (2) the subspecialty referral process, encompassing communication between physicians as well as logistical problems and delays in referrals (seven items, range 1–5, one factor, eigenvalue=2.7, Cronbach α=0.81); (3) patient characteristics, such as non-adherence to recommended follow-up (seven items, range 1–5 ,1 factor, eigenvalue=2.0, Cronbach α=0.72). For each domain, a higher score indicates a more desirable outcome.

    Analysis

    We tabulated physician characteristics and the frequency with which they reported diagnostic difficulty (our primary outcome). We then investigated whether physician characteristics and their ratings of potential barriers were associated with perceived diagnostic difficulty. We created a logistic regression model for the dependent variable of diagnostic difficulty, defined by the perception that >5% of cases in the prior year were difficult to diagnose. We chose this cut-off for frequent diagnostic difficulty based on the distribution of responses in our data (figure 1). Variables significant at the level of p=0.05 in univariate models were retained in the multivariate model, and geographic region was included as prespecified. We suspected that those reporting more frequent diagnostic difficulty would respond differently to questions about which specific factors served as barriers to timely and accurate diagnosis. Therefore, we also assessed how the overall pattern of responses to individual items in each domain differed between those reporting more versus less frequent diagnostic difficulty. We obtained IRB approval to analyse these data for publication.

    Figure 1
    Figure 1

    Diagnostic difficulty: distribution of perceived frequency of difficult to diagnose cases.

    Results

    Of 1817 surveys mailed, 1054 were completed (response rate 58%). We used American Association for Public Opinion Research response rate definition 6.17 For subsequent analysis, we restricted our study sample to 848 (80%) respondents who reported primarily practicing in the outpatient setting and had a patient panel. Respondents had been in practice for a mean of 13 years, and 32% reported prior work experience in private practice/non-group-model health maintenance organisation. Their mean age was 43, and 62% were male subjects. Overall, 50% of respondents reported that >5% of their cases were difficult to diagnose (figure 1).

    Regarding specific knowledge and judgement issues contributing to missed and delayed diagnoses, less than half of the respondents to this question (198 of 453) reported data that involved any cognitive factors; others reported system and patient factors instead. We report the most 10 common cognitive factors and the 10 most common systems factors described by participants in table 1.

    View this table:
    Table 1

    Cognitive factors related to missed and delayed diagnosis (N=251 responses from 198 respondents)

    The most common cognitive issues reported were inadequate knowledge base and inadequate detection or perception of a clinical problem. Problems with interpretation of clinical information were also frequently reported, such as mis-identification of a symptom or sign or overestimation of the reliability or saliency of a finding. Systems issues related to information availability and the negative impact of a heavy workload on the diagnostic process. Two patient-related factors emerged consistently from the responses: (1) lack of adherence to physician recommendations and diagnostic evaluation and (2) failure to provide information about symptoms.

    A variety of themes emerged in open-ended suggestions to reduce diagnostic delays, as shown in table 2. The most common suggestions related to physician workload, which included lengthening patient visit times, reducing the size of each physician's panel and off-loading non-clinical work to non-physicians. Respondents cited wait times for diagnostic tests and for results to be conveyed to physicians and patients as an area for improvement, mirroring results from other clinical settings.18 Respondents also reported that improved access to specialists would reduce diagnostic delays.

    View this table:
    Table 2

    Open-ended responses to improve diagnosis (total responses=384)

    As shown in table 3, the only physician characteristic associated with reporting more frequent diagnostic difficulty was time since completion of residency (those with more experience reported less frequent diagnostic difficulty). In multivariate analyses, diagnostic difficulty was associated with worse ratings of information processing and more problems with the referral process.

    View this table:
    Table 3

    Factors associated with higher frequency of diagnostic difficulty

    Within the domain of information processing, those who reported a higher frequency of diagnostic difficulty reported that clinically relevant information was not always available. In addition, they reported more frequent concerns about the adequacy of time to integrate clinical information and to think carefully about diagnostic issues (box 1).

    Box 1

    Items associated with reporting more diagnostic difficulty

    Information processing item

    • Incomplete, missing or illegible documentation.*

    • Test result tracking system that fails to let me know when test results are available.

    • Not receiving ED discharge summaries or consult notes when my patients have been seen in the ED or by another service.

    • Insufficient time for me to integrate the volume of clinical information (eg, labs, test results) I receive.*

    • Insufficient time for me to think carefully about diagnostic issues.*

    Referral scale item

    • Specialists return referrals with requests for additional information.*

    • Test results or diagnostic information but no recommendation for next steps from the specialist who has done the consult.*

    • Unclear understanding between specialists and me concerning who is responsible for informing and treating patients when abnormal test results are reported.*

    • Unclear understanding when a specialist should refer a patient to another specialist rather than back to me.*

    • Long wait times for specialty care consult appointments.*

    • Specialists deny referrals.*

    • Specialists not responding to my calls or emails in a timely manner.*

    Responses: 1=All of the time, 2=Most of the time, 3=Some of the time, 4=A little of the time, 5=None of the time.

    *p Value for difference in response for those reporting more frequent versus less frequent diagnostic difficulty<0.01.

    In terms of the referral process, those reporting diagnostic difficulty reported less subspecialty access, inadequate communication between PCPs and subspecialists, and insufficient guidance from subspecialists back to the referring provider (box 1). In addition, those reporting difficulty were significantly more likely to report an unclear understanding between specialists and themselves regarding who is responsible for informing and treating patients when abnormal test results are reported (box 1).

    Conclusions

    We report findings from a multi-centre survey conducted to support quality improvement efforts to address ambulatory diagnostic problems faced by PCPs. We elicited several cognitive factors PCPs believed to be associated with diagnostic delays, as well as strategies for improvement. We found half of the respondents reporting that more than 5% of their cases were difficult to diagnose. Though this is not a high proportion for any individual clinician, it represents many difficult diagnoses overall. In our analysis, diagnostic difficulty appears to correlate with inadequate time to process diagnostic information and insufficient guidance from subspecialists.

    Our study provides rich data regarding processes of care that make diagnosis vulnerable to errors in the primary care setting. Cognitive factors including inadequate knowledge base and faulty perception/detection were noted, but even when specifically asked about cognitive factors, respondents referenced system and patient factors, underscoring their importance. The most frequent cognitive factor reported was related to inadequate knowledge, in contrast to a previous study where knowledge problems were only infrequently related to diagnostic errors.16 This was followed by faulty detection or perception of diagnostic information (such as from history/exam/record review), a factor also prominent in a recent survey of paediatricians.19 In addition to improving information synthesis and reducing errors from biases and heuristics, efforts to reduce diagnostic errors should also focus on these two important factors. As with prior studies,4 ,7 lack of timely information review was commonly cited as a contributor to missed and delayed diagnosis.

    We also report specific improvement strategies that PCPs believe will improve missed and delayed diagnosis, for which empiric literature is generally lacking. Respondents' strategies for improvement often centred on physician workload; they identified several distinct links between physician workload and delays in diagnosis. First, they perceived longer visit time with patients, which may allow for more thorough history and physical examination, as important to the diagnostic process. Second, they specifically cited reducing workload in order to have protected time to review patient results to ensure follow-up of clinical abnormalities. Other recent studies also demonstrate this to be a high-risk area despite improved information availability.20 ,21 For instance, studies reveal suboptimal follow-up of abnormal test results, even in the presence of electronic health record (EHR)-based clinician notification, suggesting that that missed or delayed diagnoses are not always related to lack of access to information. Other system and cognitive factors including time and workload are likely in play and need to be targeted for improvement.

    In addition, participants suggested more time for physicians to think carefully about difficult cases, and to perform ‘cognitive work’.22 Moreover, they cited the burden associated with non-visit-related tasks. These tasks, such as following up patient test results, are increasingly becoming recognised as a significant contributor to physician workload23 and will require specific, workflow targeted efforts to address. It is possible that the patient-centred medical home model,24 with panel-based rather than visit-based reimbursement,25 will allow for clinicians to allocate their time in a way that better supports timely, correct diagnoses. As an example, in the integrated health system in which this study took place, assessment of PCP performance is shifting to quality, service and access metrics across a PCP's patient population, rather than the measurement of the number of visits per day provided by the PCP.

    Despite the presence of an integrated healthcare system where subspecialists were readily accessible, we found that the referral process is perceived as a barrier to timely and complete diagnosis. The PCPs in this study who reported diagnostic difficulty acknowledged problems accessing and communicating with subspecialists. This problem is likely to be worse in individual practices, where PCPs interact with subspecialists from several other healthcare systems using many types of communication methods.26 In addition, the issue of shared responsibility for a patient between a subspecialist and a PCP merits further enquiry. For instance, there is no clear consensus among clinicians about who should refer patients to a second specialist if the first specialty referral does not result in a diagnosis and/or treatment plan.27 Similarly, physician perceptions vary about who should inform patients about abnormal test results; studies suggest that usual clinic practice leaves many patients uninformed.18 Some degree of standardisation of referral expectations and practices is likely to enhance the diagnostic process.26

    Our study has several limitations. First, although conducted across five distinct US geographic regions with significant variability in practice settings, it was originally designed as a quality improvement project within a single health system with salaried, staff physicians and thus, its generalisability might be limited. For instance, it might not be representative of communication between PCPs and subspecialists in US ambulatory healthcare in general where per-consultation financial incentives for subspecialists are different.28 Second, our study was performed in 2005. Findings remain relevant despite the age of the data because: (a) cognitive factors described here are unlikely to have substantively altered and (b) system factors such as short visit length and problems processing information from multiple inputs are likely to persist. However, this study was conducted prior to comprehensive EHR implementation at this health system. Barriers we found associated with perceived difficulty might not necessarily be mitigated by EHRs29 and most US primary care practices do not currently have a comprehensive EHR. Third, this study was conducted prior to many of the landmark studies on diagnostic error and diagnostic processes; thus, the phrasing of the items might not at times follow taxonomies laid out in these papers.4 ,7 ,16 ,30 However, the issues described in the survey are interpretable, have been described in the literature and emanate from primary care workflow. Fourth, the patient population cared for by the PCPs, although demographically diverse,31 represents a group with health insurance, mostly provided by employers. As such, patients in this system may have fewer barriers than in other ambulatory healthcare settings. Fifth, our results reflect only the missed diagnoses that providers are aware of. There are likely other missed diagnoses that remain unknown to primary care providers, and our study cannot shed light on this type of missed diagnoses. Finally, as with all survey research, subjects who chose to respond to the survey are likely to differ from those who do not. We did not have sufficient demographic information on non-responders to make inferences about how non-response affected the reported results.

    In summary, a significant proportion of PCPs report diagnostic difficulty involving at least 5% of their patients. Misdiagnosis relates to several system and cognitive factors but knowledge gaps might be more often responsible than previously estimated. Interventions that address practice level issues such as time to process diagnostic information and better subspecialty input might potentially reduce diagnostic safety concerns in primary care.

    References

    1. ↵
      1. Bishop TF,
      2. Ryan AM,
      3. Casalino LP
      . Paid malpractice claims for adverse events in inpatient and outpatient settings. JAMA 2011;305:2427–31.
      OpenUrlCrossRefPubMed
    2. ↵
      1. Leape LL,
      2. Brennan TA,
      3. Laird N,
      4. et al
      . The nature of adverse events in hospitalized patients. Results Harv Med Pract Study N Engl J Med 1991;324:377–84.
      OpenUrl
    3. ↵
      1. Thomas EJ,
      2. Studdert DM,
      3. Burstin HR,
      4. et al
      . Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38:261–71.
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Schiff GD,
      2. Hasan O,
      3. Kim S,
      4. et al
      . Diagnostic error in medicine: analysis of 583 physician-reported errors. Arch Intern Med 2009;169:1881–7.
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Graber M
      . Diagnostic errors in medicine: a case of neglect. Jt Comm J Qual Patient Saf 2005;31:106–13.
      OpenUrlPubMed
    6. ↵
      1. Newman-Toker DE,
      2. Pronovost PJ
      . Diagnostic errors–the next frontier for patient safety. JAMA 2009;301:1060–2.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Gandhi TK,
      2. Kachalia A,
      3. Thomas EJ,
      4. et al
      . Missed and delayed diagnoses in the ambulatory setting: a study of closed malpractice claims. Ann Intern Med 2006;145:488–96.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Phillips RL Jr.,
      2. Bartholomew LA,
      3. Dovey SM,
      4. et al
      . Learning from malpractice claims about negligent, adverse events in primary care in the United States. Qual Saf Health Care 2004;13:121–6.
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Thomas EJ,
      2. Petersen LA
      . Measuring errors and adverse events in health care. J Gen Intern Med 2003;18:61–7.
      OpenUrlCrossRefPubMedWeb of Science
    10. ↵
      1. Manwell LB,
      2. Williams ES,
      3. Babbott S,
      4. et al
      . Physician perspectives on quality and error in the outpatient setting. WMJ 2009;108:139–44.
      OpenUrlPubMed
    11. ↵
      1. Kostopoulou O,
      2. Delaney BC,
      3. Munro CW
      . Diagnostic difficulty and error in primary care–a systematic review. Fam Pract 2008;25:400–13.
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Schiff GD
      . Minimizing diagnostic error: the importance of follow-up and feedback. Am J Med 2008;121:S38–42.
      OpenUrlPubMed
    13. ↵
      1. Beatty P
      . The dynamics of cognitive interviewing. In: Presser S, Rothgeb JM, Couper MP, et al., eds. Methods For Testing And Evaluating Survey Questionnaires. Hoboken, NJ: John Wiley & Sons, Inc, 2004:45–66.
    14. ↵
      1. Carbone ET,
      2. Campbell MK,
      3. Honess-Morreale L
      . Use of cognitive interview techniques in the development of nutrition surveys and interactive nutrition messages for low-income populations. J Am Diet Assoc 2002;102:690–6.
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Collins D
      . Pretesting survey instruments: an overview of cognitive methods. Qual Life Res 2003;12:229–38.
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Graber ML,
      2. Franklin N,
      3. Gordon R
      . Diagnostic error in internal medicine. Arch Intern Med 2005;165:1493–9.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      The American Association for Public Opinion Research. Standard Definitions: Final Dispositions Of Case Codes and Outcome Rates For Surveys. 7th edn. Deerfield, IL: AAPOR, 2011.
    18. ↵
      1. Casalino LP,
      2. Dunham D,
      3. Chin MH,
      4. et al
      . Frequency of failure to inform patients of clinically significant outpatient test results. Arch Intern Med 2009;169:1123–9.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Singh H,
      2. Thomas EJ,
      3. Wilson L,
      4. et al
      . Errors of diagnosis in pediatric practice: a multisite survey. Pediatrics 2010;126:70–9.
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Singh H,
      2. Thomas EJ,
      3. Mani S,
      4. et al
      . Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential? Arch Intern Med 2009;169:1578–86.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Singh H,
      2. Thomas EJ,
      3. Sittig DF,
      4. et al
      . Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain? Am J Med 2010;123:238–44.
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Singh H,
      2. Petersen LA,
      3. Thomas EJ
      . Understanding diagnostic errors in medicine: a lesson from aviation. Qual Saf Health Care 2006;15:159–64.
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Doerr E,
      2. Galpin K,
      3. Jones-Taylor C,
      4. et al
      . Between-visit workload in primary care. J Gen Intern Med 2010;25:1289–92.
      OpenUrlCrossRefPubMed
    24. ↵
      Patient-Centered Primary Care Collaborative. http://www.pcpcc.net/ (accessed 9 May 2012).
    25. ↵
      1. Rittenhouse DR,
      2. Shortell SM
      . The patient-centered medical home: will it stand the test of health reform? JAMA 2009;301:2038–40.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      1. Singh H,
      2. Esquivel A,
      3. Sittig DF,
      4. et al
      . Follow-up actions on electronic referral communication in a multispecialty outpatient setting. J Gen Intern Med 2011;26:64–9.
      OpenUrlCrossRefPubMed
    27. ↵
      1. Kirschner N,
      2. Greenlee MC
      . American College of Physicians. The Patient-Centered Medical Home Neighbor: The Interface of the Patient-Centered Medical Home with Specialty/Subspecialty Practices. Position paper. Philadelphia: American College of Physicians, 2010. http://www.acponline.org/advocacy/where_we_stand/policy/pcmh_neighbors.pdf (accessed 9 May 2012).
    28. ↵
      1. Pearson SD
      . Principles of generalist-specialist relationships. J Gen Intern Med 1999;14(Suppl 1):S13–20.
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Singh H,
      2. Graber M
      . Reducing diagnostic error through medical home-based primary care reform. JAMA 2010;304:463–4.
      OpenUrlCrossRefPubMed
    30. ↵
      1. Singh H,
      2. Thomas E,
      3. Khan M,
      4. et al
      . Identifying diagnostic errors in primary care using an electronic screening algorithm. Arch Intern Med 2007;167:302–8.
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      1. Krieger N
      . Overcoming the absence of socioeconomic data in medical records: validation and application of a census-based methodology. Am J Public Health 1992;82:703–10.
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Funding Dr Sarkar is supported by the Agency for Healthcare Research and Quality K08 HS017594. Dr. Singh is supported by an NIH K23 career development award (K23CA125585), and in part by the Houston VA HSR&D Center of Excellence (HFP90-020). Manuscript contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH or any of the other funders. None of the funders had any role in design and conduct of the study; collection, management, analysis or interpretation of the data; or preparation, review or approval of the manuscript.

    • Competing interests None.

    • Ethics approval University of California, San Francisco.

    • Provenance and peer review Not commissioned; internally peer reviewed.