Abstract
There are three potential sources of information for evaluating a clinician’s performance: documentation, patient report, and directly observed care. Current measures draw on just two of these: data recorded in the medical record and surveys of patients. Neither captures an array of performance characteristics, including clinician attention to symptoms and signs while taking a history or conducting a physical exam, accurate recording in the medical record of information obtained during the encounter, evidence based communication strategies for preventive care counseling, and effective communication behavior. Unannounced Standardized Patients (USPs) have been widely deployed as a research strategy for systematically uncovering significant performance deficits in each of these areas, but have not been adopted for quality improvement. Likely obstacles include concerns about the ethics of sending health professionals sham patients, the technical challenges of the subterfuge, and concerns about the relatively small sample sizes and substantial costs involved. However, the high frequency of significant and remediable performance deficits unmasked by USPs, and the potential to adapt registration and record keeping systems to accommodate their visits, suggest that their selective and purposeful deployment could be a cost effective and powerful strategy for addressing a gap in performance measurement.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
National Center for Quality Assurance. 2013, at http://www.ncqa.org/HEDISQualityMeasurement/PerformanceMeasurement.aspx. Accessed March 2014.
Luck J, Peabody JW. Using standardised patients to measure physicians’ practice: validation study using audio recordings. BMJ. 2002;325:679.
Glassman PA, Luck J, O’Gara EM, Peabody JW. Using standardized patients to measure quality: evidence from the literature and a prospective study. Jt Comm J Qual Improv. 2000;26:644–53.
Dresselhaus TR, Luck J, Peabody JW. The ethical problem of false positives: a prospective evaluation of physician reporting in the medical record. J Med Ethics. 2002;28:291–4.
Dresselhaus TR, Peabody JW, Lee M, Wang MM, Luck J. Measuring compliance with preventive care guidelines: standardized patients, clinical vignettes, and the medical record. J Gen Intern Med. 2000;15:782–8.
Krane NK, Anderson D, Lazarus CJ, et al. Physician practice behavior and practice guidelines: using unannounced standardized patients to gather data. J Gen Intern Med. 2009;24:53–6.
Hornung RL, Hansen LA, Sharp LK, Poorsattar SP, Lipsky MS. Skin cancer prevention in the primary care setting: assessment using a standardized patient. Pediatr Dermatol. 2007;24:108–12.
Gorter S, van der Heijde DM, van der Linden S, et al. Psoriatic arthritis: performance of rheumatologists in daily practice. Ann Rheum Dis. 2002;61:219–24.
Weiner SJ, Schwartz A, Weaver F, et al. Contextual errors and failures in individualizing patient care: a multicenter study. Ann Intern Med. 2010;153:69–75.
Finn A. Shopper benchmarking of durable-goods chains and stores. J Serv Res. 2001;3:310–20.
Levine M. CEJA Report 3-A-08 Secret Shopper “Patients.”. Chicago, IL: American Medical Association; 2008.
Franz CE, Epstein R, Miller KN, et al. Caught in the act? Prevalence, predictors, and consequences of physician detection of unannounced standardized patients. Health Serv Res. 2006;41:2290–302.
Speedie SM, Niewoehner C. The Minnesota Virtual Clinic: using a simulated EMR to teach medical students basic science and clinical concepts. AMIA Annual Symposium proceedings / AMIA Symposium AMIA Symposium 2003:1013.
van Zanten M, Boulet JR, McKinley D. Using standardized patients to assess the interpersonal skills of physicians: six years’ experience with a high-stakes certification examination. Health Commun. 2007;22:195–205.
Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA. 2007;298:993–1001.
Whelan GP, Boulet JR, McKinley DW, et al. Scoring standardized patient examinations: lessons learned from the development and administration of the ECFMG Clinical Skills Assessment (CSA). Med Teach. 2005;27:200–6.
Schwartz A, Weiner SJ, Weaver F, et al. Uncharted territory: measuring costs of diagnostic errors outside the medical record. BMJ Qual Saf 2012.
Audet AM, Doty MM, Shamasdin J, Schoenbaum SC. Measure, learn, and improve: physicians’ involvement in quality improvement. Health Aff (Millwood). 2005;24:843–53.
Funding/Support
Drs. Weiner and Schwartz’s research on unannounced standardized patients has been supported by the Veterans Affairs, Health Services Research and Development, and the National Board of Medical Examiners.
Disclaimer
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs or the United States government.
Disclosures
The authors are co-principals of the Institute for Practice and Provider Performance Improvement, founded to employ unannounced standardized patient assessments as a quality improvement service. They have not at this point received any fees or contracts. There are no other relationships or activities that could appear to have influenced the submitted work.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Weiner, S.J., Schwartz, A. Directly Observed Care: Can Unannounced Standardized Patients Address a Gap in Performance Measurement?. J GEN INTERN MED 29, 1183–1187 (2014). https://doi.org/10.1007/s11606-014-2860-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11606-014-2860-7