You are here

Article Text

Article menu
Original research
Association of National Hospital Quality Measure adherence with long-term mortality and readmissions
Free
  1. David M Shahian1,2,
  2. Gregg S Meyer2,3,4,
  3. Elizabeth Mort2,3,4,
  4. Susan Atamian4,
  5. Xiu Liu4,
  6. Andrew S Karson2,3,4,
  7. Lawrence D Ramunno5,
  8. Hui Zheng6
  1. 1Center for Quality and Safety and Department of Surgery, Massachusetts General Hospital, Boston, USA
  2. 2Harvard Medical School, Boston, USA
  3. 3Department of Medicine, Massachusetts General Hospital, Boston, USA
  4. 4Center for Quality and Safety, Massachusetts General Hospital, Boston, USA
  5. 5Northeast Health Care Quality Foundation, Dover, USA
  6. 6Biostatistics Center, Massachusetts General Hospital, Boston, USA
  1. Correspondence to Dr David M Shahian, Center for Quality and Safety and Department of Surgery, Massachusetts General Hospital, 55 Fruit St., Boston, MA 02114, USA; dshahian{at}partners.org

Abstract

Background In existing studies, the association between adherence with recommended hospital care processes and subsequent outcomes has been inconsistent. This has substantial implications because process measure scores are used for accountability, quality improvement and reimbursement. Our investigation addresses methodological concerns with previous studies to better clarify the process–outcomes association for three common conditions.

Methods The study included all patients discharged from Massachusetts General Hospital between 1 July 2004 and 31 December 2007 with a principle diagnosis of acute myocardial infarction (AMI), heart failure (HF) or pneumonia (PN) who were eligible for at least one National Hospital Quality Measure. The number of patients analysed varied by measure (374 to 3020) depending on Centers for Medicare and Medicaid Services eligibility criteria. Hospital data were linked with state administrative data to determine mortality and readmissions. For patients with multiple admissions, the time-weighted impact of measure failures on mortality was estimated using exponential decay functions. All patients had follow-up for at least 1 year or until death or readmission. Cox models were used to estimate HRs adjusted for transfer status, age, gender, race, census block-group socioeconomic status, number of Elixhauser comorbidities, and do not resuscitate orders.

Results Adjusted survival and freedom from readmission for AMI and PN showed superior results for 100% and 50–99% adherence compared with 0–49% adherence. For HF, the results were inconsistent and sometimes paradoxical, although several individual measures (eg, ACE inhibitor/angiotensin receptor blockade) were associated with improved outcomes.

Conclusion Adherence with recommended AMI and PN care processes is associated with improved long-term outcomes, whereas the results for HF measures are inconsistent. The evidence base for all process measures must be critically evaluated, including the strength of association between these care processes and outcomes in real-world populations. Some currently recommended processes may not be suitable as accountability measures.

  • Process measure
  • adherence
  • survival
  • readmission
  • clinical practice guidelines
  • control charts
  • run charts
  • mortality (standardised mortality ratios)
  • performance measures
  • quality measurement
  • diabetes mellitus
  • qualitative research

https://doi.org/10.1136/bmjqs-2011-000615

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Performance measurement and reporting are central features of healthcare reform, and hospitals devote considerable resources to optimising their scores on these measures. Because of their implications for accountability, referrals and reimbursement, performance measures should be based on the highest level of evidence. Important evaluation criteria include the quality, consistency and quantity of the aggregate evidence base, and high magnitude of net benefit.1

    Outcomes measures are the preferred modality for assessing performance because they integrate the net impact of measured and unmeasured processes and structures of care,2 an important principle noted by Donabedian3 nearly half a century ago. However, for many diagnoses, accurate estimation of outcomes is challenging because of small sample sizes, infrequent adverse outcomes, unavailable or reliable data, and inadequate risk models to account for patient severity. Because of these limitations, process and structure measures have been used to assess performance for certain diagnoses, either alone or in combination with outcomes. Some of these measures are used for public reporting, including the Centers for Medicare and Medicaid Services (CMS) Hospital Compare4 National Hospital Quality Measures (NHQM) for acute myocardial infarction (AMI), heart failure (HF) and pneumonia (PN).

    Process measures used to assess provider performance should have a demonstrable, proximate association with important outcomes such as mortality and readmission. Unfortunately, studies of these associations have had inconsistent and inconclusive results, in some instances raising serious concerns as to the suitability of such measures for provider profiling. For example, Werner and Bradlow5 studied AMI, HF and PN and found limited association of hospital-level, individual process measure adherence and hospital mortality at 30 days and 1 year.

    Studies of AMI or acute coronary syndromes have provided differing results depending on timing of the endpoint and hospital versus patient-level outcomes.6–8 Overall, the evidence base for AMI process performance measures is relatively strong, particularly for some measures such as timeliness of revacularisation (eg, ‘door to balloon time’).9 However, many such studies have focused on short-term mortality and have not investigated the potential association of measure adherence with longer-term mortality or readmissions.

    Studies of PN, the other acute NHQM, have shown consistently positive associations between measures such as antibiotic selection and short-term (eg, hospital or 30-day) mortality.10–22 Recent evidence also suggests that, like AMI, the full impact of PN and its treatment may not be apparent until patients are observed for a longer time period,23 and this has not been done in most process–outcomes studies. It is also difficult to assess the efficacy of inpatient measures such as smoking cessation counselling that may be confounded by post-discharge patient compliance.24

    Among the three diseases, studies of HF have demonstrated the most problematic link between process measure adherence and outcomes.25 26 As a chronic disease with frequent readmissions, longer-term follow-up is important, and the results of current studies raise questions. Using the OPTIMIZE-HF registry, Fonarow and colleagues27 found no association of American College of Cardiology/American Heart Association endorsed HF performance measures with in-hospital mortality. Patterson and colleagues28 found similar results in a study of hospital-level measure adherence and 1-year mortality and readmissions among Medicare recipients, although both studies were limited by the lack of socioeconomic information, a potentially important confounder. In the study by Fonarow and colleagues,27 discharge ACE inhibitor/angiotensin receptor blockade (ACEI/ARB) use was associated with decreased 60–90-day mortality and readmission, similar to the findings in previous studies of long-term ACEI use.29 Discharge β blockade, not a performance measure because of concerns regarding its use in potentially unstable hospitalised patients,30 was also associated with reduced mortality; similar findings have been observed for other emerging HF care processes.31 A few studies do show a process adherence–outcomes association. Kfoury and colleagues found a dose–response association between the number of HF processes used and 1-year mortality,32 and several studies suggest that adherence with recommended care processes correlated with lower readmission rates.33–35

    Validating the directionality and strength of the process–outcomes association is critical. If process adherence does not improve patient outcomes, then scarce resources have been misallocated, and stakeholders may be misinformed about quality of care. The motivation for this study was to expand our understanding of this issue and to address some of the methodological concerns with prior investigations. Many of these studies are based on older data; there was substantial heterogeneity among institutions from which data were aggregated; follow-up duration was inadequate; populations were not inclusive; aggregate hospital results rather than patient-level data were analysed; and potentially important confounders (eg, socioeconomic status) were unavailable.

    We explored the process–outcomes association using contemporary data from a large urban tertiary medical center that serves both as a referral center and as the community hospital for a diverse local population. A robust institutional database provided numerous variables unavailable in most administrative and clinical registries. It was thus possible to adjust for these factors and to better isolate the association between process adherence and outcomes.

    Methods

    Study population

    This observational retrospective cohort study analysed patients discharged from the Massachusetts General Hospital (MGH) between 1 July 2004 and 31 December 2007 with a diagnosis of AMI, HF or PN who were eligible for at least one NHQM. Not all patients who qualified for study entry were eligible for each measure. Therefore, denominators for various measures vary according to CMS eligibility criteria.

    Principle outcomes

    The main outcomes are all-cause mortality and all-cause readmissions at 90 days and 1 year after hospital discharge. We obtained death and readmission information on each patient for at least 1-year follow-up and until 31 December 2008. For the mortality cohort, survival time was defined as the interval between the last eligible discharge that included a particular measure and either patient death or achievement of 90-day or 1-year survival, whichever was shorter. We chose discharge rather than admission date because we were focusing on hospital survivors and long-term outcomes, and because many of the measures studied were provided at discharge. Sensitivity analyses were also performed which calculated survival time beginning at hospital admission for measures provided on arrival. For the readmission cohort, freedom from readmission was the period between the first discharge and subsequent readmissions or the 90-day/1-year endpoints, whichever was shorter. Sensitivity analysis was performed with outcome assessed from date of admission rather than discharge for arrival measures.

    Patients in the mortality study cohort could have multiple discharges, each of which was regarded as having a time-dependent impact on outcomes (see below). Only one primary discharge per patient was included for the readmission cohort. Subsequent admissions were considered as readmissions, and only the first readmission was included in the study.

    Exclusions

    Patients who died before hospital discharge were excluded from the study cohort (49 patients with AMI, 25 with PN and none with HF), as were out-of-state patients because their death and readmissions could not be reliably ascertained. For the mortality endpoint, sensitivity analysis was performed in which we included patients who died in hospital.

    Further exclusions were applied to the readmission cohort based on CMS readmission model specifications. We excluded patients who were discharged to another short-term general hospital or left against medical advice. For the AMI readmission cohort, patients were excluded if they stayed in hospital for only 1 day (CMS regards such patients as unlikely to have had AMI) and if they also had not been transferred from another acute hospital. The latter criterion is our addition to the CMS rule, reflecting our institutional experience. MGH and other referral centers often receive transfer patients for tertiary evaluation or treatment who have a length of stay of 1 day, but who are nonetheless eligible for at least one AMI measure.

    Readmissions within 30 days for percutaneous coronary interventions (PCIs) or coronary artery bypass graft were not counted as AMI readmissions because they are typically planned. For the HF discharge instruction measure, we limited the study cohort to patients who were discharged to home with or without services (having discharge destination code of 01 and 06), thus mitigating any potential confounding by the care rendered at extended care facilities (13 patients excluded from the study cohort).

    Data sources and linkages

    Study patients were identified from an MGH registry that included patient demographic and clinical characteristics and measure adherence results. This registry included patient identifiers that allowed us to link measure adherence with Massachusetts state mortality and readmission data. Mortality data from 2004 to 2008 were obtained from Massachusetts Vital Records and Statistics. We merged MGH adherence data with state death records by patient name and date of birth. If a match was not found in the state mortality database, we assumed the patient was still alive at the end of the study.

    For the readmission cohort, we obtained Massachusetts all-hospital inpatient data from the Massachusetts Division of Health Care Finance and Policy. These data included a unique state-level patient identifier, the admitting hospital and medical record number, admission and discharge dates, and principle and secondary diagnoses. We first merged MGH internal data with the state database to extract the unique patient identifiers, matching by MGH medical record number and admission and discharge dates. Using this approach, 99% of MGH study admissions were matched to the state inpatient database. We then used the unique patient identifier to identify subsequent readmissions from the state inpatient database. Following this initial linking, we stripped unique patient identifiers and all other patient identifiers from the analysis dataset.

    Measure aggregation and extent of adherence

    At the individual measure level, the term adherence in this study signifies that a patient was reported to CMS as having met the NHQM requirements for a given measure that were applicable at the time of their hospitalisation. As the intent of the study was to estimate the association between measure non-adherence and various outcomes, measure failure or non-adherence was coded 1, and adherence was coded 0. The extent of measure adherence was estimated for each discharge using a variety of techniques.

    Mortality method 1—impact of any previous measure failure

    For mortality, we included all discharges for a given patient for each specific diagnosis. We estimated the association of measure non-adherence with mortality using two different approaches. In the first method, for each individual measure, we estimated the mortality HRs associated with failure on any hospitalisation. Similarly, we estimated the HRs associated with failure to achieve 100% adherence with all components of a group of related measures (eg, arrival measures, discharge measures), as defined in online table 1, on any admission. Finally, we also estimated an all-or-none score, the most stringent approach. To receive credit, there could be no failures on any measure for which the patient was eligible, on any admission. All-or-none scoring is commonly used, although it may be criticised on statistical grounds because the joint probability of failure increases with the number of eligible measures, which varies among patients.

    Mortality method 2—time-weighted impact of previous measure failures

    For the mortality cohort, we also used a second method to account for the fact that some patients had multiple admissions within the study period for the same diagnosis. We hypothesised that the impact on survival of each individual failure decreased exponentially over time. Patients with failure on all admissions would have a score of exactly 1, and adherence on a distant admission with process measure failure on all subsequent admissions would result in a score that approached 1. Conversely, a single remote admission with a measure failure followed by numerous subsequent admissions with perfect adherence would have a score close to 0.

    We estimated continuous, time-weighted average adherence scores for multiple admission patients using exponential decay functions (online appendix 1). Online appendix 2 demonstrates how we used this approach to estimate the time-weighted average scores. Average time-weighted adherence scores were then used as independent variables in Cox models. The resulting adjusted HRs estimate the association between each 10% increase in average time-weighted adherence score (where failure=1 and adherence=0) and subsequent mortality. We further categorised all patients into three groups based on their average time-weighted adherence scores to estimate dose–response relationships.

    Readmission method

    For the readmission cohort, only the first readmission was studied. Subsequent readmissions were disregarded, and it was therefore unnecessary to use time-weighted decay. We studied the association of readmission with non-adherence to individual, group and all-or-none measure (1=measure non-adherence, 0=adherence). We also analysed three categories of adherence, based on the number of adherent measures divided by the number of eligible measures.

    Statistical analysis

    In the primary analysis, we calculated patient demographic and clinical characteristics and performance for all measures. We used the log-rank test to identify potential confounders and a multivariable Cox proportional hazards model to adjust for confounding effects. The final model covariates included the patient's age, gender, race (white, other), census block-group socioeconomic status (SES, lowest 25th vs other) using the multi-indicator approach of Diez-Roux and colleagues,36 the number of Elixhauser comorbidities, do not resuscitate order on admission for arrival measure models, do not resuscitate order anytime during hospitalisation for other measure models, and transfer to MGH for non-arrival measures (CMS rules regard transfer patients as ineligible for arrival measures). Adjusted HRs and 95% CIs were computed for various time points. Cox models were fit separately for individual, group and composite measure outcomes at 90 days and 1 year for all-cause mortality and readmissions.

    All statistical analyses were performed with SAS software, V.9.2.

    Human subject protection

    This research was reviewed and approved by the MGH/Partners Institutional Review Board. Use of level VI Massachusetts state data was conducted in conformity with all applicable state regulations.

    Results

    Online table 1 lists the percentage adherence for each individual, group and overall measure for the three conditions. Individual measure adherence ranged from 54.8% (influenza vaccine assessment) to 100% (O2 assessment for PN). For each condition, online table 2 lists the bivariate demographic characteristics, clinical features, outcomes (mortality and readmissions), and p values for patients with 100% adherence to all measures versus <100% adherence.

    Table 1 shows the 90-day and 365-day adjusted mortality HRs for non-adherence to individual, group and all-or-none measures.

    View this table:
    Table 1

    Association of measure non-adherence with subsequent all-cause mortality

    For AMI, failure on the arrival aspirin or β-blocker measures were independently associated with significantly reduced 1-year survival. Arrival measures, discharge measures and the all-or-none measures were all strongly associated with increased risk of death at 90 days and 365 days. After excluding in-hospital deaths, there were insufficient patients to analyse the association of PCI <90 min and late outcomes. Sensitivity analyses (using in-hospital deaths and time from admission rather than discharge) changed the absolute values of most HRs but not their directionality or statistical significance (results available upon request). However, with in-hospital deaths included, failure on the PCI <90 min measure was a highly significant predictor of 90-day (adjusted HR 5.48, 95% CI 1.22 to 24.65) and 365-day mortality (HR 3.62, 95% CI 1.07 to 12.28). HRs for the time-weighted approach (table 1) are directionally similar for most predictors, although in some cases they change from statistically significant to non-significant; the time-weighted categorical variable shows a dose–response relationship. Sensitivity analyses, as described previously, also showed that failure to achieve PCI <90 min was a significant predictor of 90-day and 365-day mortality.

    For PN, the other acute condition, only failure on the blood culture timing variable had a significant association with 90-day and 365-day survival. Time-weighted average non-adherence was associated with a marginally increased risk of 365-day mortality and the 0–49% adherence group had shorter survival time. Sensitivity analyses (including in-hospital deaths and calculating survival time from admission rather than discharge) did not reveal substantial changes in the associations of process adherence and survival for arrival measures.

    For HF, non-adherence with the ACEI/ARB measure was associated with an increased hazard of long-term mortality, as was failure on the smoking counselling measure. The results for the time-weighted measures were generally insignificant and did not show a dose–response association.

    Table 2 shows the association between measure non-adherence and 90-day and 1-year readmission.

    View this table:
    Table 2

    Association of non-adherence and subsequent all-cause readmissions

    For AMI, failure on the PCI <90 min measure was a strong predictor of readmission for both time-periods. Failure on the smoking counselling measure was associated with a greater risk of 90-day readmission. Failure on the discharge group measure and all-or-none measure were also associated with an increased adjusted risk of readmission, as was <50% measure adherence. Sensitivity analyses using time from admission to readmission changed only the absolute value but not the directionality or statistical significance of the results, with one exception: failure to achieve PCI<90 min was still strongly associated with 365-day readmission (adjusted HR 2.33, 95% CI 1.33 to 4.06) but the results at 90 days were no longer significant.

    For HF, failure on the ACEI/ARB measure and the discharge instruction measure were each associated with an increased risk of 90-day and 1-year readmission. Failure to receive all components of the Patient Information Measure group (online table 1) was associated with increased risk of 1-year readmission, as was 50–99% categorical measure adherence (but not 0–49% adherence) compared with 100% adherence. Finally, for PN, non-adherence with the antibiotic selection and smoking counselling measures was associated with increased risk of readmission, as was 0–49% (vs 100%) overall adherence. Associations between arrival measure adherence and readmission were not substantially changed in sensitivity analyses using time from admission (rather than discharge) to readmission.

    Figure 1A,B present the adjusted survival and freedom from readmission curves for AMI. In both instances the curves for 100% and 50–99% adherence were virtually superimposable, whereas <50% adherence was associated with significantly lower survival and higher probability of readmission.

    Figure 1
    Figure 1

    Adjusted 1-year survival (A) and freedom from readmissions (B) by measure adherence for patients with acute myocardial infarction (AMI).

    Figure 2A,B depict survival and freedom from readmission for HF, and the results are paradoxical. Adherence of 0–49% has the best survival, full adherence has intermediate results (not statistically different from 0–49%), and 50–99% adherence has the worst survival. Figure 2B shows that 100% adherence and 0–49% are nearly superimposable and associated with the highest freedom from readmission, whereas 50–99% adherence has significantly lower freedom from readmission.

    Figure 2
    Figure 2

    Adjusted 1-year survival (A) and freedom from readmissions (B) by measure adherence for patients with heart failure (HF).

    The results for PN (figure 3A,B) are similar to those for AMI. Survival and readmission results for 100% and 50–99% adherence are similar and superior, whereas 0–49% adherence is associated with significantly inferior results for both.

    Figure 3
    Figure 3

    Adjusted 1-year survival (A) and freedom from readmissions (B) by measure adherence for patients with pneumonia.

    Discussion

    Injudicious selection of performance measures may misallocate scarce performance improvement resources, divert attention from more efficacious processes of care, inappropriately penalise or reward providers and misinform consumers.25 37–40 If not associated with substantial positive effects, efforts to optimise scores on some publicly reported measures may even have a net negative impact because of adverse unintended consequences such as premature activation of cardiac catheterisation labs in AMI41 42 or administration of antibiotics to patients before a diagnosis of PN has been firmly established.38 39 Masoudi40 suggested that the net positive and unintended negative consequences be assessed by ‘balancing measures’.

    Numerous explanations have been hypothesised for the absent, weak or inconsistent association between process measures and outcomes. For example, even efficacious care processes may explain only a small proportion of variation in patient outcomes.7 Some recommended processes may have been selected based on limited observational studies, or conversely they may have been chosen based on randomised trials with rigid eligibility and exclusion criteria. Results from the latter may not generalise well to broader, real-world patient populations.5 25 28 43 44 The value of some process measures may be offset by their unintended adverse consequences.37 38 40–42 For this reason, some processes such as β blockade for HF that favourably impact outcomes with chronic use are not included in current inpatient measure sets, as their use may be associated with higher risk in hospitalised patients.30 45 Some processes (eg, cardiac resynchronisation therapy or implantable defibrillators) may have substantial utility in selected subgroups but their inclusion/exclusion criteria and potential adverse consequences may preclude broad use as performance measures.46–49 Failure to individualise the application of guidelines could produce inferior results by encouraging inappropriate care of specific patients.39 50–52

    Inconclusive results from existing process–outcomes studies may also result from flaws in study design, documentation or follow-up. These include the inability to adjust for important clinical, socioeconomic or hospital confounders8 25 26 28 53–63; ceiling effects for some ‘topped-out’ measures5 7; inadequate duration of follow-up to detect significant differences in outcomes attributable to process adherence (eg, smoking counselling, anticoagulation for atrial fibrillation)7 26 32; dissociation between care prescribed at discharge and subsequent patient adherence24 28 64 65; ‘check-the-box’ mentality, while failing to conform with the ‘spirit’ of the measure28; and inaccurate hospital documentation that does not accurately reflect care delivered.28

    Contributions of the current study

    Our study addresses important methodological concerns with previous investigations of the process–outcomes association. We use contemporary, all age and payer, patient-level data from a single large institution with a diverse patient population. This mitigates concerns regarding between-hospital heterogeneity. We have access to detailed, census block-group socioeconomic data,36 a potentially important confounder that is lacking in most studies and that may impact post-discharge patient behaviour. Patients of low SES may be less likely to consistently use prescribed medications or attend outpatient clinics, thus adversely influencing survival and readmission rates, irrespective of the efficacy of inpatient care processes.

    Our study also investigated process measure adherence at several levels of aggregation. Performance on one measure does not necessarily correlate with performance on other measures,8 42 and the degree to which an entire ‘bundle’ of measures is necessary to achieve the optimal effect is uncertain. Finally, our study employs a novel statistical approach, exponential decay, to address the problem of measure adherence or failure on repeated admissions for the same patient.

    Our results suggest that even when in-hospital mortalities are excluded, process measure adherence (individual and group) for two common acute conditions, AMI and PN, is often positively associated with long-term outcomes. Furthermore, these outcomes improve with higher degrees of process adherence. Presumably, aspirin and β blockade on arrival limit myocardial ischaemia and the subsequent extent of infarcted myocardium, thereby reducing early mortality. However, the extent of infarct also has an association with late outcomes. When appropriate medications are not given and the resulting infarct is larger, more extensive scar formation and post-infarct remodelling lead to a higher incidence of late deaths and readmissions from HF and arrhythmias. By a similar mechanism, failure to open the infarct artery in less than 90 min may be associated with increased in-hospital mortality and late outcomes.

    The association between failure on the PN blood culture timing measure and long-term outcomes is interesting. Blood cultures drawn prior to antibiotic administration may allow more rapid determination of the causative organism in the most severely ill patients, and early institution of tailored, specific antibiotic therapy may favourably influence late outcomes. However, it is also possible that this process measure mainly reflects other unmeasured aspects of care, such as the use of standardised protocols.

    For HF, only ACEI/ARB adherence and smoking counselling were associated with lower mortality and readmission. For the various composites, the results were generally inconsistent and insignificant, emphasising the importance of validating entire measure bundles and their individual constituent measures. Currently endorsed HF measures may lack a strong association with outcomes, or unmeasured outpatient factors may outweigh the importance of adherence with inpatient care processes in this chronic condition.

    Limitations

    We believe our single institutional study cohort is advantageous for the reasons noted previously, but it does raise the issue of generalisability, despite our diverse community and referral population.

    We were unable to document compliance with and changes to prescribed care regimes following discharge. This potential unmeasured confounder may be most relevant for HF, a chronic illness requiring systematic outpatient follow-up.66 67

    Because we limited the study of the HF discharge instruction measure to those patients discharged home, our results may not be applicable to patients discharged to extended care facilities.

    The current study is observational, and there may be unmeasured confounders. These limitations should be considered in evaluating the strength of any causal inferences derived from our analyses.

    Conclusions

    In aggregate, these findings demonstrate the potential limitations of using process measures to assess provider performance. If appropriately selected, as in the case of AMI and PN, measure adherence may promote the fundamental goal of profiling—to improve short-term and long-term patient outcomes. Conversely, in our study only some of the current HF measures are favourably associated with long-term outcomes, and there is no overall dose–response association. This may reflect fundamental issues with the selected process measures, the impact of unmeasured processes of care or the confounding effect of subsequent outpatient care. Focusing on measures without a strong evidence base may divert scarce quality improvement resources, encourage marginally effective care practices and misclassify providers.

    As performance measurement evolves, the emphasis should increasingly shift to direct outcomes metrics, including mortality and morbidity, patient-reported outcomes and satisfaction. When process measures are used, their association with outcomes should ideally be validated in randomised trials and real-world observational studies. Some recommended care processes may not be suitable as accountability measures.

    References

    1. ↵
    2. ↵
      1. Porter ME
      . What is value in health care? N Engl J Med 2010;363:2477–81.
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Donabedian A
      . Evaluating the quality of medical care. Milbank Mem Fund Q 1966;44:166–206.
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      CMS Hospital Compare. http://www.hospitalcompare.hhs.gov/ (accessed 23 Apr 2011).
    5. ↵
      1. Werner RM,
      2. Bradlow ET
      . Relationship between Medicare's hospital compare performance measures and mortality rates. JAMA 2006;296:2694–702.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Eagle KA,
      2. Montoye CK,
      3. Riba AL,
      4. et al
      . Guideline-based standardized care is associated with substantially lower mortality in Medicare patients with acute myocardial infarction: the American College of Cardiology's Guidelines Applied in Practice (GAP) Projects in Michigan. J Am Coll Cardiol 2005;46:1242–8.
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Bradley EH,
      2. Herrin J,
      3. Elbel B,
      4. et al
      . Hospital quality for acute myocardial infarction: correlation among process measures and relationship with short-term mortality. JAMA 2006;296:72–8.
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Peterson ED,
      2. Roe MT,
      3. Mulgund J,
      4. et al
      . Association between hospital process performance and outcomes among patients with acute coronary syndromes. JAMA 2006;295:1912–20.
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Antman EM,
      2. Anbe DT,
      3. Armstrong PW,
      4. et al
      . ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 1999 Guidelines for the Management of Patients with Acute Myocardial Infarction). Circulation 2004;110:e82–292.
      OpenUrlFREE Full Text
    10. ↵
      1. Gleason PP,
      2. Meehan TP,
      3. Fine JM,
      4. et al
      . Associations between initial antimicrobial therapy and medical outcomes for hospitalized elderly patients with pneumonia. Arch Intern Med 1999;159:2562–72.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Menéndez R,
      2. Ferrando D,
      3. Vallés JM,
      4. et al
      . Influence of deviation from guidelines on the outcome of community-acquired pneumonia. Chest 2002;122:612–17.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Dudas V,
      2. Hopefl A,
      3. Jacobs R,
      4. et al
      . Antimicrobial selection for hospitalized patients with presumed community-acquired pneumonia: a survey of nonteaching US community hospitals. Ann Pharmacother 2000;34:446–52.
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Arnold FW,
      2. Summersgill JT,
      3. Lajoie AS,
      4. et al
      . A worldwide perspective of atypical pathogens in community-acquired pneumonia. Am J Respir Crit Care Med 2007;175:1086–93.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Frei CR,
      2. Restrepo MI,
      3. Mortensen EM,
      4. et al
      . Impact of guideline-concordant empiric antibiotic therapy in community-acquired pneumonia. Am J Med 2006;119:865–71.
      OpenUrlCrossRefPubMed
    15. ↵
      1. Bodi M,
      2. Rodriguez A,
      3. Solé-Violán J,
      4. et al
      . Antibiotic prescription for community-acquired pneumonia in the intensive care unit: impact of adherence to Infectious Diseases Society of America guidelines on survival. Clin Infect Dis 2005;41:1709–16.
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Martinez R,
      2. Reyes S,
      3. Lorenzo MJ,
      4. et al
      . Impact of guidelines on outcome: the evidence. Semin Respir Crit Care Med 2009;30:172–8.
      OpenUrlCrossRefPubMed
    17. ↵
      1. Menendez R,
      2. Torres A,
      3. Zalacain R,
      4. et al
      . Guidelines for the treatment of community-acquired pneumonia: predictors of adherence and outcome. Am J Respir Crit Care Med 2005;172:757–62.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Mortensen EM,
      2. Restrepo M,
      3. Anzueto A,
      4. et al
      . Effects of guideline-concordant antimicrobial therapy on mortality among patients with community-acquired pneumonia. Am J Med 2004;117:726–31.
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Sharpe BA
      . Guideline-recommended antibiotics in community-acquired pneumonia: not perfect, but good. Arch Intern Med 2009;169:1462–4.
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Malone DC,
      2. Shaban HM
      . Adherence to ATS guidelines for hospitalized patients with community-acquired pneumonia. Ann Pharmacother 2001;35:1180–5.
      OpenUrlAbstract/FREE Full Text
    21. ↵
      1. McCabe C,
      2. Kirchner C,
      3. Zhang H,
      4. et al
      . Guideline-concordant therapy and reduced mortality and length of stay in adults with community-acquired pneumonia: playing by the rules. Arch Intern Med 2009;169:1525–31.
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Dean NC,
      2. Bateman KA,
      3. Donnelly SM,
      4. et al
      . Improved clinical outcomes with utilization of a community-acquired pneumonia guideline. Chest 2006;130:794–9.
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Bordon J,
      2. Wiemken T,
      3. Peyrani P,
      4. et al
      . Decrease in long-term survival for hospitalized patients with community-acquired pneumonia. Chest 2010;138:279–83.
      OpenUrlCrossRefPubMed
    24. ↵
      1. Reeves GR,
      2. Wang TY,
      3. Reid KJ,
      4. et al
      . Dissociation between hospital performance of the smoking cessation counseling quality metric and cessation outcomes after myocardial infarction. Arch Intern Med 2008;168:2111–17.
      OpenUrlCrossRefPubMed
    25. ↵
      1. Fonarow GC,
      2. Peterson ED
      . Heart failure performance measures and outcomes: real or illusory gains. JAMA 2009;302:792–4.
      OpenUrlCrossRefPubMed
    26. ↵
      1. Bonow RO
      . Measuring quality in heart failure: do we have the metrics? Circ Cardiovasc Qual Outcomes 2008;1:9–11.
      OpenUrlFREE Full Text
    27. ↵
      1. Fonarow GC,
      2. Abraham WT,
      3. Albert NM,
      4. et al
      . Association between performance measures and clinical outcomes for patients hospitalized with heart failure. JAMA 2007;297:61–70.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Patterson ME,
      2. Hernandez AF,
      3. Hammill BG,
      4. et al
      . Process of care performance measures and long-term outcomes in patients hospitalized with heart failure. Med Care 2010;48:210–16.
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Flather MD,
      2. Yusuf S,
      3. Køber L,
      4. et al
      . Long-term ACE-inhibitor therapy in patients with heart failure or left-ventricular dysfunction: a systematic overview of data from individual patients. ACE-Inhibitor Myocardial Infarction Collaborative Group. Lancet 2000;355:1575–81.
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Bonow RO,
      2. Bennett S,
      3. Casey DE Jr.,
      4. et al
      . ACC/AHA clinical performance measures for adults with chronic heart failure: a report of the American College of Cardiology/American Heart Association Task Force on Performance Measures (Writing Committee to Develop Heart Failure Clinical Performance Measures): endorsed by the Heart Failure Society of America. Circulation 2005;112:1853–87.
      OpenUrlFREE Full Text
    31. ↵
      1. Hernandez AF,
      2. Hammill BG,
      3. Peterson ED,
      4. et al
      . Relationships between emerging measures of heart failure processes of care and clinical outcomes. Am Heart J 2010;159:406–13.
      OpenUrlCrossRefPubMedWeb of Science
    32. ↵
      1. Kfoury AG,
      2. French TK,
      3. Horne BD,
      4. et al
      . Incremental survival benefit with adherence to standardized heart failure core measures: a performance evaluation study of 2958 patients. J Card Fail 2008;14:95–102.
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Polanczyk CA,
      2. Newton C,
      3. Dec GW,
      4. et al
      . Quality of care and hospital readmission in congestive heart failure: an explicit review process. J Card Fail 2001;7:289–98.
      OpenUrlCrossRefPubMed
    34. ↵
      1. Chung ES,
      2. Lin G,
      3. Casey DE Jr.,
      4. et al
      . Relationship of a quality measure composite to clinical outcomes for patients with heart failure. Am J Med Qual 2008;23:168–75.
      OpenUrlAbstract/FREE Full Text
    35. ↵
      1. VanSuch M,
      2. Naessens JM,
      3. Stroebel RJ,
      4. et al
      . Effect of discharge instructions on readmission of hospitalised patients with heart failure: do all of the Joint Commission on Accreditation of Healthcare Organization's heart failure core measures reflect better care? Qual Saf Health Care 2006;15:414–17.
      OpenUrlAbstract/FREE Full Text
    36. ↵
      1. Diez-Roux AV,
      2. Kiefe CI,
      3. Jacobs DR Jr.,
      4. et al
      . Area characteristics and individual-level socioeconomic position indicators in three population-based epidemiologic studies. Ann Epidemiol 2001;11:395–405.
      OpenUrlCrossRefPubMedWeb of Science
    37. ↵
      1. Wachter RM
      . Expected and unanticipated consequences of the quality and information technology revolutions. JAMA 2006;295:2780–3.
      OpenUrlCrossRefPubMedWeb of Science
    38. ↵
      1. Wachter RM,
      2. Flanders SA,
      3. Fee C,
      4. et al
      . Public reporting of antibiotic timing in patients with pneumonia: lessons from a flawed performance measure. Ann Intern Med 2008;149:29–32.
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. Bratzler DW,
      2. Nsa W,
      3. Houck PM
      . Performance measures for pneumonia: are they valuable, and are process measures adequate? Curr Opin Infect Dis 2007;20:182–9.
      OpenUrlPubMedWeb of Science
    40. ↵
      1. Masoudi FA
      . Measuring the quality of primary PCI for ST-segment elevation myocardial infarction: time for balance. JAMA 2007;298:2790–1.
      OpenUrlCrossRefPubMedWeb of Science
    41. ↵
      1. Larson DM,
      2. Menssen KM,
      3. Sharkey SW,
      4. et al
      . ‘False-positive’ cardiac catheterization laboratory activation among patients with suspected ST-segment elevation myocardial infarction. JAMA 2007;298:2754–60.
      OpenUrlCrossRefPubMed
    42. ↵
      1. Wang TY,
      2. Fonarow GC,
      3. Hernandez AF,
      4. et al
      . The dissociation between door-to-balloon time improvement and improvements in other acute myocardial infarction care processes and patient outcomes. Arch Intern Med 2009;169:1411–19.
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      1. Campbell AR,
      2. Satran D,
      3. Larson DM,
      4. et al
      . ST-elevation myocardial infarction: which patients do quality assurance programs include? Circ Cardiovasc Qual Outcomes 2009;2:648–55.
      OpenUrlAbstract/FREE Full Text
    44. ↵
      1. Rathore SS,
      2. Wang Y,
      3. Radford MJ,
      4. et al
      . Quality of care of Medicare beneficiaries with acute myocardial infarction: who is included in quality improvement measurement? J Am Geriatr Soc 2003;51:466–75.
      OpenUrlCrossRefPubMedWeb of Science
    45. ↵
      1. Jessup M,
      2. Abraham WT,
      3. Casey DE,
      4. et al
      . 2009 focused update: ACCF/AHA guidelines for the diagnosis and management of heart failure in adults: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines: developed in collaboration with the International Society for Heart and Lung Transplantation. Circulation 2009;119:1977–2016.
      OpenUrlFREE Full Text
    46. ↵
      1. Stevenson LW
      . Counting performance with therapies for heart failure: aiming for quality or quantity? Circulation 2010;122:561–6.
      OpenUrlFREE Full Text
    47. ↵
      1. Ghali JK,
      2. Massie BM,
      3. Mann DL,
      4. et al
      . Guidelines, performance measures, and the practice of medicine: mind the gap. J Card Fail 2010;16:915–18.
      OpenUrlCrossRefPubMed
    48. ↵
      1. Kerr EA,
      2. Hofer TP,
      3. Hayward RA,
      4. et al
      . Quality by any other name? A comparison of three profiling systems for assessing health care quality. Health Serv Res 2007;42:2070–87.
      OpenUrlCrossRefPubMedWeb of Science
    49. ↵
      1. Kerr EA,
      2. Smith DM,
      3. Hogan MM,
      4. et al
      . Building a better quality measure: are some patients with ‘poor quality’ actually getting good care? Med Care 2003;41:1173–82.
      OpenUrlCrossRefPubMedWeb of Science
    50. ↵
      1. Rubin HR,
      2. Pronovost P,
      3. Diette GB
      . The advantages and disadvantages of process-based measures of health care quality. Int J Qual Health Care 2001;13:469–74.
      OpenUrlAbstract/FREE Full Text
    51. ↵
      1. Chen JL,
      2. Sosnov J,
      3. Lessard D,
      4. et al
      . Impact of do-not-resuscitation orders on quality of care performance measures in patients hospitalized with acute heart failure. Am Heart J 2008;156:78–84.
      OpenUrlCrossRefPubMedWeb of Science
    52. ↵
      1. Eddy DM,
      2. Adler J,
      3. Patterson B,
      4. et al
      . Individualized guidelines: the potential for increasing quality and reducing costs. Ann Intern Med 2011;154:627–34.
      OpenUrlCrossRefPubMedWeb of Science
    53. ↵
      1. Hammill BG,
      2. Curtis LH,
      3. Fonarow GC,
      4. et al
      . Incremental value of clinical data beyond claims data in predicting 30-day outcomes after heart failure hospitalization. Circ Cardiovasc Qual Outcomes 2011;4:60–7.
      OpenUrlAbstract/FREE Full Text
    54. ↵
      1. Hausmann LR,
      2. Ibrahim SA,
      3. Mehrotra A,
      4. et al
      . Racial and ethnic disparities in pneumonia treatment and mortality. Med Care 2009;47:1009–17.
      OpenUrlCrossRefPubMedWeb of Science
    55. ↵
      1. Birkmeyer NJ,
      2. Gu N,
      3. Baser O,
      4. et al
      . Socioeconomic status and surgical mortality in the elderly. Med Care 2008;46:893–9.
      OpenUrlCrossRefPubMedWeb of Science
    56. ↵
      1. Bernheim SM,
      2. Spertus JA,
      3. Reid KJ,
      4. et al
      . Socioeconomic disparities in outcomes after acute myocardial infarction. Am Heart J 2007;153:313–19.
      OpenUrlCrossRefPubMedWeb of Science
    57. ↵
      1. Rathore SS,
      2. Masoudi FA,
      3. Wang Y,
      4. et al
      . Socioeconomic status, treatment, and outcomes among elderly patients hospitalized with heart failure: findings from the National Heart Failure Project. Am Heart J 2006;152:371–8.
      OpenUrlCrossRefPubMedWeb of Science
    58. ↵
      1. Peterson E,
      2. Yancy CW
      . Eliminating racial and ethnic disparities in cardiac care. N Engl J Med 2009;360:1172–4.
      OpenUrlCrossRefPubMedWeb of Science
    59. ↵
      1. Ayanian JZ,
      2. Weissman JS,
      3. Chasan-Taber S,
      4. et al
      . Quality of care by race and gender for congestive heart failure and pneumonia. Med Care 1999;37:1260–9.
      OpenUrlCrossRefPubMedWeb of Science
    60. ↵
      1. Popescu I,
      2. Vaughan-Sarrazin MS,
      3. Rosenthal GE
      . Differences in mortality and use of revascularization in black and white patients with acute MI admitted to hospitals with and without revascularization services. JAMA 2007;297:2489–95.
      OpenUrlCrossRefPubMedWeb of Science
    61. ↵
      1. Chang WC,
      2. Kaul P,
      3. Westerhout CM,
      4. et al
      . Effects of socioeconomic status on mortality after acute myocardial infarction. Am J Med 2007;120:33–9.
      OpenUrlCrossRefPubMedWeb of Science
    62. ↵
      1. Selby JV,
      2. Swain BE,
      3. Gerzoff RB,
      4. et al
      . Understanding the gap between good processes of diabetes care and poor intermediate outcomes: Translating Research into Action for Diabetes (TRIAD). Med Care 2007;45:1144–53.
      OpenUrlCrossRefPubMedWeb of Science
    63. ↵
      1. Masoudi FA,
      2. Peterson ED,
      3. Anderson JL,
      4. et al
      . Clinical guidelines and performance measures responsible guidance and accountability. J Card Fail 2010;16:919–21.
      OpenUrlCrossRefPubMed
    64. ↵
      1. Ho PM,
      2. Magid DJ,
      3. Shetterly SM,
      4. et al
      . Medication nonadherence is associated with a broad range of adverse outcomes in patients with coronary artery disease. Am Heart J 2008;155:772–9.
      OpenUrlCrossRefPubMedWeb of Science
    65. ↵
      1. Jackevicius CA,
      2. Li P,
      3. Tu JV
      . Prevalence, predictors, and outcomes of primary nonadherence after acute myocardial infarction. Circulation 2008;117:1028–36.
      OpenUrlAbstract/FREE Full Text
    66. ↵
      1. Ducharme A,
      2. Doyon O,
      3. White M,
      4. et al
      . Impact of care at a multidisciplinary congestive heart failure clinic: a randomized trial. CMAJ 2005;173:40–5.
      OpenUrlAbstract/FREE Full Text
    67. ↵
      1. Hernandez AF,
      2. Greiner MA,
      3. Fonarow GC,
      4. et al
      . Relationship between early physician follow-up and 30-day readmission among Medicare beneficiaries hospitalized for heart failure. JAMA 2010;303:1716–22.
      OpenUrlCrossRefPubMedWeb of Science

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Funding Internal.

    • Competing interests Dr Ramunno is Chief Quality Officer for Northeast Health Care Foundation, a Medicare QIO. He has been involved with the development, revision, and implementation of National Hospital Quality Measures as a contractor with the federal government.

    • Patient consent Retrospective review of previously collected administrative claims data and state all-payer administrative records. Not feasible to obtain permission, and risk considered minimal.

    • Ethics approval Partners IRB (2008P000003).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Online supplemental material includes detailed information on numbers of eligible patients for each measure; descriptive characteristics and bivariate associations; and examples of the exponential decay approach used in this study.