The practice of emergency medicine/original researchThe Presence of Outcome Bias in Emergency Physician Retrospective Judgments of the Quality of Care
Introduction
In 1980, Donadebian1 proposed the triad of structure, process, and outcome to characterize the quality of medical care and argued that process measures were essential because “the most direct route to an assessment of the quality of care is an examination of that care.” Despite 30 years of contentious arguments about the relative merits of process and outcome measures of quality, whether the perspective is medical quality, legal culpability, or cost-effectiveness, process measures indisputably have an important role in judging the quality of medical care.
With regard to certain measures (eg, was the patient with acute myocardial infarction given an aspirin?), judging the process of care may be straightforward. However, as judgments become more subjective (eg, did the physician appreciate the laxity of the anterior cruciate ligament?), they become more vulnerable to a series of biases. The first of these is the human tendency to reduce complex processes to individual errors in cognition.2 The second is that context can bias raters; a physician may be reluctant to find fault with a colleague's care, whereas a plaintiff's attorney may be more than willing to do so. Third, knowledge of the patient's outcome may impede a rater's ability to judge process objectively. Outcome bias, the propensity to assign blame more readily when the outcome is bad, has been described in radiology, anesthesiology, internal medicine, and surgery.3, 4, 5, 6, 7
In certain peer review and medicolegal situations, the biased assessment of quality of the process of care can have profound negative emotional, occupational, and financial consequences for the treating physician who may be inappropriately blamed for morbidity or mortality beyond his or her control. Given that assessments of process are more often triggered by the occurrence of a bad outcome than by the occurrence of a good one, and given the high-risk nature of emergency medicine, we thought it important to determine the extent to which outcome bias can alter assessments of the quality of process of care in emergency medicine. We designed a study to determine to what extent knowledge of outcome affects emergency physicians' ability to assess the quality of the process of care, and what factors affect degree of outcome bias.
Section snippets
Materials and Methods
We performed a Web-based survey of members of the American Academy of Emergency Medicine. A list of potential participants was generated by academy staff, who subsequently sent out an e-mail solicitation for participation. The e-mail informed subjects that they had been randomly selected to aid in a study of quality assurance. Participants were then given a hyperlink to 6 brief clinical scenarios and asked to judge the quality of care provided.
All scenarios were created by the authors, and each
Results
We had originally planned to send our request to a sample of American Academy of Emergency Medicine members, but because of a miscommunication the request for participation, in the form of a single e-mail, was sent to all 4,785 members. After 2 weeks, we checked the database and, finding 622 responses, stopped enrollment. We dropped 35 incomplete sessions, leaving 587 complete cases for evaluation. Eighty-five percent of respondents were men. Thirty-two percent were younger than 40 years, 34%
Limitations
This study has a number of limitations, the most notable being that we used hypothetic scenarios instead of actual cases. Furthermore, to reduce respondent burden we provided brief synopses of the cases rather than actual medical records. It is possible that respondents given a complete record would interpret the facts differently. For example, when given a case with a bad outcome, a rater might scrutinize the entire record, searching for clues that should have alerted the physician that the
Discussion
Whether care was good, bad, or indeterminate, retrospective assessments of the quality of the process of care should be unaffected by the patient's outcome. Unfortunately, those who judge the quality of care are seldom unaware of that outcome, and our study shows that emergency physicians' judgments of the process of care are affected by their knowledge of the outcome. In cases of intermediate quality, outcome bias shifts ratings by a magnitude equivalent to 1 qualitative step in quality (eg,
References (10)
- et al.
Replacing hindsight with insight: toward better understanding of diagnostic failures
Ann Emerg Med
(2007) Methods for deriving criteria for assessing the quality of medical care
Med Care Rev
(1980)Outcome bias
AJR Am J Roentgenol
(2004)- et al.
Incidence of adverse events and negligence in hospitalized patientsResults of the Harvard Medical Practice Study I
N Engl J Med
(1991) - et al.
Relation between negligent adverse events and the outcomes of medical-malpractice litigation
N Engl J Med
(1996)
Cited by (0)
Supervising editor: Donald M. Yealy, MD
Dr. Yealy was the supervising editor on this article. Dr. Schriger did not participate in the editorial review or decision to publish this article.
Author contributions: MG and JAT conceived of the study. MG, DLS, and JAT designed the study. MG and DLS performed the data collection and statistical analyses. MG drafted the article, and all authors contributed substantially to its revision. MG was responsible for data collection and analysis and takes responsibility for the paper as a whole.
Funding and support: By Annals policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article that might create any potential conflict of interest. See the Manuscript Submission Agreement in this issue for examples of specific conflicts covered by this statement. Dr. Schriger is funded in part by an unrestricted grant from the Korein Foundation.
Publication dates: Available online January 12, 2011.
Earn CME Credit: Continuing Medical Education is available for this article at: www.ACEP-EMedhome.com.
Reprints not available from the authors.
Please see page 324 for the Editor's Capsule Summary of this article.
Provide feedback on this article at the journal's Web site, www.annemergmed.com.
A podcast for this article is available at www.annemergmed.com.