Skip to main content

Advertisement

Log in

Measuring patients’ experiences with individual primary care physicians

Results of a statewide demonstration project

  • Original Articles
  • Published:
Journal of General Internal Medicine Aims and scope Submit manuscript

Abstract

BACKGROUND: Measuring and reporting patients’ experiences with health plans has been routine for several years. There is now substantial interest in measuring patients’ experiences with individual physicians, but numerous concerns remain.

OBJECTIVE: The Massachusetts Ambulatory Care Experiences Survey Project was a statewide demonstration project designed to test the feasibility and value of measuring patients’ experiences with individual primary care physicians and their practices.

DESIGN: Cross-sectional survey administered to a statewide sample by mail and telephone (May–August 2002).

PATIENTS: Adult patients from 5 commercial health plans and Medicaid sampled from the panels of 215 generalist physicians at 67 practice sites (n=9,625).

MEASUREMENTS: Ambulatory Care Experiences Survey produces 11 summary measures of patients’ experiences across 2 domains: quality of physician-patient interactions and organizational features of care. Physician-level reliability was computed for all measures, and variance components analysis was used to determine the influence of each level of the system (physician, site, network organization, plan) on each measure. Risk of misclassifying individual physicians was evaluated under varying reporting frameworks.

RESULTS: All measures except 2 achieved physician-level reliability of at least 0.70 with samples of 45 patients per physician, and several exceeded 0.80. Physicians and sites accounted for the majority of system-related variance on all measures, with physicians accounting for the majority on all “interaction quality” measures (range: 61.7% to 83.9%) and sites accounting for the largest share on “organizational” measures (range: 44.8% to 81.1%). Health plans accounted for neglible variance (<3%) on all measures. Reporting frameworks and principles for assuring misclassification risk ≤ 2.5% were identified.

CONCLUSIONS: With considerable national attention on the importance of patient-centered care, this project demonstrates the feasibility of obtaining highly reliable measures of patients’ experiences with individual physicians and practices. The analytic findings underscore the validity and importance of looking beyond health plans to individual physicians and sites as we seek to improve health care quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Epstein AM. Public release of performance data: a progress report from the front. JAMA. 2000;283:1884–6.

    Article  PubMed  CAS  Google Scholar 

  2. Lee TH, Meyer GS, Brennan TA. A middle ground on public accountability. N Engl J Med. 2004;350:2409–12.

    Article  PubMed  CAS  Google Scholar 

  3. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the Twenty-First Century. Washington, DC: National Academy Press; 2001.

    Google Scholar 

  4. Solomon LS, Zaslavsky AM, Landon BE, Cleary PD. Variation in patient-reported quality among health care organizations. Health Care Fin Rev. 2002;23:1–16.

    Google Scholar 

  5. Landon BE, Wilson IB, Cleary PD. A conceptual model of the effects of health care organizations on the quality of medical care. JAMA. 1998;279:1377–82.

    Article  PubMed  CAS  Google Scholar 

  6. Rodriguez H, von Glahn T, Rogers W, Chang H, Fanjiang G, Safran D. Evaluating patients’ experiences with individual physicians: a randomized trial of mail, internet and interactive voice response (IVR) telephone administration of surveys. Med Care. 2006, in press.

  7. Safran DG, Kosinski M, Tarlov AR, et al. The primary care assessment survey: tests of data quality and measurement performance. Med Care. 1998;36:728–39.

    Article  PubMed  CAS  Google Scholar 

  8. Hays RD, Shaul JA, Williams VS, et al. Psychometric properties of the CAHPS 1.0 survey measures. Consumer Assessment of Health Plans Study. Med Care. 1999;37:22–31.

    Article  Google Scholar 

  9. Institute of Medicine. Primary Care: America’s Health in a New Era. Washington, DC: National Academy Press; 1996.

    Google Scholar 

  10. Dillman DA. Mail and Telephone Surveys: The Total Design Method. New York: John Wiley; 1978.

    Google Scholar 

  11. Fowler FJ Jr., Gallagher PM, Stringfellow VL, Zaslavsky AM, Thompson JW, Cleary PD. Using telephone interviews to reduce nonresponse bias to mail surveys of health plan members. Med Care. 2002;40:190–200.

    Article  PubMed  Google Scholar 

  12. Zaslavsky AM, Zaborski LB, Cleary PD. Factors affecting response rates to the consumer assessment of health plans study survey. Med Care. 2002;40:485–99.

    Article  PubMed  Google Scholar 

  13. Lasek RJ, Barkley W, Harper DL, Rosenthal GE. An evaluation of the impact of nonresponse bias on patient satisfaction surveys. Med Care. 1997;35:646–52.

    Article  PubMed  CAS  Google Scholar 

  14. Little R, Rubin D. Statistical Analysis with Missing Data. NY: Wiley and Sons; 1987.

    Google Scholar 

  15. Allen HM, Rogers WH. The consumer health plan value survey: round two. Health Aff (Millwood). 1997;16:156–66.

    Article  Google Scholar 

  16. Brenner H. Alternative approaches for estimating prevalence in epidemiologic surveys with two waves of respondents. Am J Epidemiol. 1995;142:1236–45.

    PubMed  CAS  Google Scholar 

  17. Herbert R, Bravo G, Korner-Bitensky N, Voyer L. Refusal and information bias associated with postal questionnaires and face-to-face interviews in very elderly subjects. J Clin Epidemiol. 1996;49:373–81.

    Article  Google Scholar 

  18. Keeter S, Miller C, Kohut A, Groves RM, Presser S. Consequences of reducing nonresponse in a national telephone survey. Public Opin Q. 2000;64:125–48.

    Article  PubMed  CAS  Google Scholar 

  19. Gasquet I, Falissard B, Ravaud P. Impact of reminders and method of questionnaire distribution on patient response to mail-back satisfaction survey. J Clin Epidemiol. 2001;54:1174–80.

    Article  PubMed  CAS  Google Scholar 

  20. Friedman EM, Clusen NA, Hartzell M. Better Late? Characteristics of Late Responders to a Health Care Survey. No. PP03-52, 1–7. Washington, DC: Mathematica Policy Research, Inc; 2004.

    Google Scholar 

  21. Nunnelly J, Bernstein I. Psychometric Theory. 3rd ed. New York: McGraw-Hill; 1994.

    Google Scholar 

  22. SAS Institute Inc. SAS/STAT User’s Guide. Version 8. Cary, NC: SAS Institute Inc.; 1999.

    Google Scholar 

  23. Zaslavsky AM, Landon BE, Beaulieu ND, Cleary PD. How consumer assessments of managed care vary within and among markets. Inquiry. 2000;37:146–61.

    PubMed  CAS  Google Scholar 

  24. Hofer TP, Hayward RA, Greenfield S, Wagner EH, Kaplan SH, Manning WG. The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease. JAMA. 1999;281:2098–105.

    Article  PubMed  CAS  Google Scholar 

  25. Landon BE, Normand SL, Blumenthal D, Daley J. Physician clinical performance assessment: prospects and barriers. JAMA. 2003;290:1183–9.

    Article  PubMed  CAS  Google Scholar 

  26. Miller ME, Hui SL, Tierney WM, McDonald CJ. Estimating physician costliness. An empirical Bayes approach. Med Care. 1993;31:YS16–28.

    Article  PubMed  CAS  Google Scholar 

  27. Orav EJ, Wright EA, Palmer RH, Hargraves JL. Issues of variability and bias affecting multisite measurement of quality of care. Med Care. 1996;34:SS87–101.

    Article  PubMed  CAS  Google Scholar 

  28. Hayward RA, Manning WG Jr., McMahon LF Jr., Bernard AM. Do attending or resident physician practice styles account for variations in hospital resource use? Med Care. 1994;32:788–94.

    Article  PubMed  CAS  Google Scholar 

  29. DiMatteo MR. Enhancing patient adherence to medical recommendations. JAMA. 1994;271:79–83.

    Article  PubMed  CAS  Google Scholar 

  30. DiMatteo MR, Sherbourne CD, Hays RD, et al. Physicians’ characteristics influence patients’ adherence to medical treatment: results from the Medical Outcomes Study. Health Psychol. 1993;12:93–102.

    Article  PubMed  CAS  Google Scholar 

  31. Francis V, Korsch BM, Morris MJ. Gaps in doctor-patient communication: patients’ response to medical advice. N Engl J Med. 1969;280:535–40.

    Article  PubMed  CAS  Google Scholar 

  32. Wilson IB, Rogers WH, Chang H, Safran DG. Cost-related skipping of medications and other treatments among Medicare beneficiaries between 1998 and 2000: results of a national study. J Gen Intern Med. 2005;20:715–20.

    Article  PubMed  Google Scholar 

  33. Safran DG, Taira DA, Rogers WH, Kosinski M, Ware JE, Tarlov AR. Linking primary care performance to outcomes of care. J Fam Pract. 1998;47:213–20.

    PubMed  CAS  Google Scholar 

  34. Greenfield S, Kaplan S, Ware JE Jr. Expanding patient involvement in care. Effects on patient outcomes. Ann Intern Med. 1985;102:520–8.

    PubMed  CAS  Google Scholar 

  35. Greenfield S, Kaplan SH, Ware JEJ, Yano EM, Frank HJ. Patients’ participation in medical care: effects on blood sugar control and quality of life in diabetes. J Gen Intern Med. 1988;3:448–57.

    Article  PubMed  CAS  Google Scholar 

  36. Stewart MA. Effective physician-patient communication and health outcomes: a review [see comments]. CMAJ. 1995;152:1423–33.

    PubMed  CAS  Google Scholar 

  37. Roter DL, Hall JA, Kern DE, Barker LR, Cole KA, Roca RP. Improving physicians’ intervewing skills and reducing patients’ emotional distress. A randomized clinical trial. Arch Intern Med. 1995;155:1877–84.

    Article  PubMed  CAS  Google Scholar 

  38. Safran DG, Montgomery JE, Chang H, Murphy J, Rogers WH. Switching doctors: predictors of voluntary disenrollment from a primary physician’s practice. J Fam Pract. 2001;50:130–6.

    PubMed  CAS  Google Scholar 

  39. Levinson W, Roter DL, Mullooly JP, Dull V, Frankel RM. Physician-patient communication: the relationship with malpractice claims among primary care physicians and surgeons. JAMA. 1997;277:553–9.

    Article  PubMed  CAS  Google Scholar 

  40. Beckman HB, Markakis KM, Suchman AL, Frankel RM. The doctor-patient relationship and malpractice: lessons from plaintiff depositions. Arch Intern Med. 1994;154:1365–70.

    Article  PubMed  CAS  Google Scholar 

  41. Hickson GBC, Clayton EW, Entman SS, et al. Obstetrician’s prior malpractice experience and patients’ satisfaction with care. JAMA. 1994;272:1583–7.

    Article  PubMed  CAS  Google Scholar 

  42. Miller S. American Board of Medical Specialties: Implementing the CAHPS Ambulatory Care Survey: A Panel Discussion of Current and Planned Activities. Presented at CAHPS Across the Health Care Continuum: 9th National User Group Meeting. Baltimore, MD, 12-2-2004.

  43. Karp M. Massachusetts Health Quality Partners: Implementing the CAHPS Ambulatory Care Survey: A Panel Discussion of Current and Planned Activities. Presented at CAHPS Across the Health Care Continuum: 9th National User Group Meeting. Baltimore, MD, 12-2-2004.

  44. Damberg C. Pacific Business Group on Health; Implementing the CAHPS Ambulatory Care Survey: A Panel Discussion of Current and Planned Activities. Presented at CAHPS Across the Health Care Continuum: 9th National User Group Meeting. Baltimore, MD, 12-2-2004.

  45. Centers for Medicare and Medicaid Services (CMS). CMS Doctor’s Office Quality Project Overview, 7-15-2004. http://www.cms.hhs.gov/quality/doq/DOQOverview.pdf

  46. Safran DG. Defining the future of primary care: what can we learn from patients? Ann Intern Med. 2003;138:248–55.

    PubMed  Google Scholar 

  47. Murphy J, Chang H, Montgomery J, Rogers WH, Safran DG. The quality of physician-patient relationships: patients’ experiences 1996–1999. J Fam Pract. 2001;50:123–9.

    PubMed  CAS  Google Scholar 

  48. National Committee for Quality Assurance. State of Managed Care Quality 2000. Washington, DC, 2000.

  49. National Committee for Quality Assurance. State of Managed Care Quality, 2001. Washington, DC, 2001.

  50. Jencks SF, Huff ED, Cuerdon T. Change in the quality of care delivered to Medicare beneficiaries, 1998–1999 to 2000–2001. JAMA. 2003;289:305–12.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Dana Gelb Safran ScD.

Additional information

The authors have no conflicts of interest to report.

This research was supported by grants from the Commonwealth Fund and the Robert Wood Johns Foundation. The authors gratefully acknowledge members of our National Advisory Committee, our Massachusetts steering committee, and our project officers (Anne-Marie Audet, M.D., Steven Shoenbaum, M.D., and Michael Rothman, Ph.D.) for their invaluable advise and guidance throughout the project period. The authors also gratefully acknowledge Paul Kallaur, Nina Smith and their project staff at the Centers for the Study of Services (CSS) their technical expertise and commitment to excellence in obtaining the data from the study sample. Finally, we thank Ira B. Wilson, M.D., M.S. for comments on an earlier draft and Jamie Belli for technical assistance in preparing this manuscript.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Safran, D.G., Karp, M., Coltin, K. et al. Measuring patients’ experiences with individual primary care physicians. J GEN INTERN MED 21, 13–21 (2006). https://doi.org/10.1111/j.1525-1497.2005.00311.x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1111/j.1525-1497.2005.00311.x

Key Words

Navigation