Original articles
Understanding Variability in Physician Ratings of the Appropriateness of Coronary Angiography After Acute Myocardial Infarction

https://doi.org/10.1016/S0895-4356(98)00166-8Get rights and content

Abstract

We examined variability in ratings of the appropriateness of coronary angiography for 890 clinical scenarios (indications) after an acute myocardial infarction (AMI) from a nine-member multispecialty panel as a function of panel characteristics and the attributes of the clinical indications. We documented a substantial degree of reliability in the ratings. However, key differences among the experts in terms of both their overall propensity to score high and their beliefs regarding the impact of clinical factors on appropriateness were identified. Age, cardiac complications, post-AMI angina, and noninvasive test results were the clinical factors most strongly related to appropriateness ratings for coronary angiography. Further research on the effectiveness of coronary angiography in older patients and in patients with shock, pulmonary edema, and silent ischemia is needed to improve our knowledge about the appropriateness of this procedure in these patients.

Introduction

Measurement of the overuse and underuse of medical procedures is an important public health activity to aid in targeting educational programs. Two decades of health services research have documented variations in the use of many medical and surgical procedures 1, 2. For example, cardiac procedures, such as coronary angiography, are less frequently performed in females, in blacks, and in rural hospitals [3]. Concerned by observations such as these, the RAND Corporation formalized a method [4] that uses expert opinion to assess the appropriateness of medical procedures and ultimately to produce explicit medical guidelines for care. These guidelines can then be used by physicians, patients, payors, and state or other regulatory agencies to improve treatment decisions and, consequently, the quality of care that is delivered.

Because, however, of the prominent roles that opinion-based guidelines serve, it is important to understand the extent to which the guidelines rely on individual raters as well as on the available clinical information. It is particularly important to identify groups of clinical indications for which expert opinion is the most divergent, so that efforts can be aimed at resolving such differences.

Appropriateness ratings for coronary angiography after an acute myocardial infarction (AMI) were updated in October of 1995 [5], in the Department of Health Care Policy, Harvard Medical School as part of a study funded by the Agency for Health Care Policy and Research. The RAND method [4], which combines evidence from the literature with expert opinion through a formal judgement process, was used. This approach has been used extensively in the past decade to produce guidelines for many medical and surgical procedures, including hysterectomy [6], coronary artery bypass graft surgery [7], percutaneous transluminal coronary angioplasty [8], upper gastrointestinal endoscopy [9], colonoscopy [10], cholecystectomy [10], carotid endarterectomy [9], and spinal manipulation [11] and has been used previously to obtain appropriateness ratings for coronary angiography 10, 12. The method consists of two stages: a comprehensive literature review from which clinical indications are developed, and the convening of an expert panel to rate the appropriateness of the indications based on the literature review and their expert judgements.

In this article, we described the distribution of appropriateness ratings for coronary angiography in patients suffering an AMI as rated by a multispecialty panel of nine medical experts. We estimated rater, indication, and measurement error components of variability in the ratings using hierarchical statistical models [13], to understand the impact of each source. We also estimated the relative importance of each clinical variable used to define the indications in determining appropriateness of angiography so that resources can be allocated toward understanding the effectiveness of this procedure in clinical subgroups for which there are divergent ratings.

Section snippets

Literature Review

A comprehensive review of the literature relevant to the effectiveness and risk of coronary angiography in AMI patients published between 1970 and 1995 was performed [5]. Articles for this review were identified by a MEDLINE search for English-language articles published during the period January 1970 to June 1995. Search terms included: myocardial-infarction, indications, angiography, angioplasty, transluminal, percutaneous coronary, thrombolytic therapy, coronary artery bypass, and

Description of the Clinical Indications for Angiography

The indications describing the appropriateness of angiography before discharge were defined by five variables: the patient’s age (75 years or older, or less than 75 years), time since symptom onset (less than 6 hours, 6 to 12 hours, more than 12 hours), prior use of thrombolytic therapy, presence of strong contraindications to thrombolytic therapy, and the presence of one of seven conditions complicating the AMI. The cardiac complications included shock, persistent chest pain, persistent

Discussion

Medical guidelines are increasingly used to understand variations in the delivery of many medical and surgical procedures. In particular, targeting geopolitical areas, provider networks, or other delivery sites who inappropriately deliver care, as measured by a guideline, is a difficult yet necessary public health mission. The most common approach to summarize evidence regarding treatment efficacy within clinical strata is the method pioneered by the RAND Corporation and the one used in this

Acknowledgements

We gratefully acknowledge two anonymous reviewers for helpful comments on an earlier version of the manuscript. This work was supported by grant R01-HS08071 from the Agency for Health Care Policy and Research, Rockville, MD.

References (20)

  • K.L. Kahn et al.

    The effect of comorbidity on appropriateness ratings for two gastrointestinal procedures

    J Clin Epidemiol

    (1988)
  • J. Wennberg et al.

    Variations in medical care among small areas

    Sci Am

    (1982)
  • Office of Research and Demonstrations, Health Care Financing Administration. Health Care Financing Administration,...
  • I.S. Udvarhelyi et al.

    Acute myocardial infarction in the Medicare populationProcess of care and clinical outcomes

    JAMA

    (1992)
  • R.H. Brook et al.

    A method for the detailed assessment of the appropriateness of medical technologies

    Int J Technol Assess Health Care

    (1986)
  • D.W. Bates et al.

    Coronary angiography and angioplasty after acute myocardial infarction

    Ann Intern Med

    (1997)
  • S.J. Berstein et al.

    The appropriateness of hysterectomyA comparison of care in seven health plans

    JAMA

    (1993)
  • S.J. Bernstein et al.

    The appropriateness of the use of cardiovascular proceduresBritish versus US perspectives

    Int J Technol Assess Health Care

    (1993)
  • L.H. Hilborne et al.

    The appropriateness of use of percutaneous transluminal coronary angioplasty in New York State

    JAMA

    (1993)
  • R.H. Brook et al.

    Predicting the appropriate use of carotid endarterectomy, upper gastrointestinal endoscopy, and coronary angiography

    N Engl J Med

    (1990)
There are more references available in the full text version of this article.

Cited by (12)

  • Role of opinion in appropriate use criteria

    2009, Journal of Cardiovascular Computed Tomography
  • An experimental study of determinants of group judgments in clinical guideline development

    2004, Lancet
    Citation Excerpt :

    Our findings support the idea that evidence is used to confirm pre-existing opinions rather than change them.28 Other studies that have assessed the effect of group composition on consensus decisions have focused on surgical conditions.12–15 By contrast with our findings, the investigators noted that judgments often reflected specialist interests, but they did not investigate whose judgments were most closely aligned with the evidence.

  • Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: A matched analysis using propensity scores

    2001, Journal of Clinical Epidemiology
    Citation Excerpt :

    After a literature review of the appropriateness of angiography [15], 890 clinical indications were specified for use of the procedure within 12 weeks of the infarction, 92 of which related to angiography during the initial hospitalization. Five variables were used to classify patients during the initial hospitalization: (1) duration of symptom onset (<6 h, 6–12 h, >12 h); (2) patient age (<75 years, ⩾75 years); (3) prior use of thrombolytic therapy; (4) presence of contraindications to thrombolytic therapy; and (5) presence of a condition complicating the infarction (shock, persistent chest pain, persistent pulmonary edema, noninvasive evidence of mitral regurgitation or ventricular septal defect, left ventricular ejection fraction <35%, stress-induced ischemia, and recurrent ventricular tachycardia or fibrillation 24 h after the infarction) [19]. The 92 clinical indications were divided approximately equally across the three time frames describing the duration of symptom onset, with 28 indications falling into the greater than 12-h time frame.

View all citing articles on Scopus
View full text