METHODS

Study design

An evaluation study was conducted in three Swiss academic medical centres. A retrospective cohort study was performed to assess whether chart based process indicators were risk factors for administrative outcome indicators (readmission) for patients admitted to hospital with HF. We found that, in patients with HF, the determination of ventricular function was associated with hospital mortality but process indicators derived from evidence based guidelines were not related to early readmission. These results have been published elsewhere.¹³ We then examined the reverse association—that is, whether readmission could be used as a measure to predict quality of care. The rationale for the direction of this analysis follows the logic of using readmission as a quality of care indicator that should predict change in the quality of process. This logic does not, however, follow a time sequence and might appear counterintuitive, but this should not be a problem because the nature of the relation examined is not time related and the measure of the dependent variable (readmission) does not influence the measure of the independent variable (guideline derived process indicators). We also assessed the validity of readmission as a quality indicator using process indicators as the gold standard.

Setting and patients

The three hospitals which participated in the study are urban public university hospitals and are the major primary care hospitals for their respective areas. All the study patients were discharged between 1 January and 31 December 1999 with a principal or secondary diagnosis of HF using the International Classification of Disease 10th revision (ICD-10) codes I50.0, I50.1, I50.9, I11.0, I13.0 and I13.2. In two of the hospitals 976 and 774 eligible patients were found, 700 of which were randomly selected in each centre based on our power calculation. In the third hospital the quality of the administrative data was not so good and we were able to select only 234 eligible patients. Patients were excluded from the study if they had died during the hospital stay or if they had left the hospital against medical advice, were transferred to another acute care facility, were discharged in 1998, or had secondary heart failure due to valvular heart disease, acute myocardial infarction, cor pulmonale, chronic renal failure, hyperthyroidosis, thiamine deficiency, amyloidosis, or chronic obstructive lung disease treated with oxygen.

Data collection

Data abstraction was conducted by trained doctors or medical record specialists. In two hospitals the entire medical charts were available for data abstraction while, in the third, only the electronic medical records which included the discharge letter, laboratory results, and all cardiological procedures were available. Demographic characteristics, risk factors, symptoms and findings at admission, and discharge details were taken from the medical records. Thirty day readmissions were calculated using administrative data. All cause readmissions were assessed and only patients from the index hospital were included but, because these hospitals are university referral centres, each for a different catchment area, we assumed that only a few patients could have been readmitted to a different hospital. Furthermore, for one provider we could establish that no patients were readmitted to another Swiss hospital using a unique identifier from the Swiss Federal Statistical Office. While we would have liked to include only unplanned readmissions in the definition, this was impossible with the database available. We were also not able to exclude from the readmission denominator patients who died outside hospital within 30 days of discharge.

Process quality indicators

Process quality indicators have been developed from evidence based guidelines^14–18 and used in previous studies.^19–21 They were adapted to the Swiss setting in collaboration with key clinicians. Table 1 summarises these quality indicators with their respective level of evidence.

The first quality indicator was the proportion of patients with HF who had a measurement of left ventricular function (LVF). LVF was identified in the chart by the presence of a previously measured ejection fraction by echocardiography, cardiac catheterisation, radionuclide ventriculography, or by a narrative statement in the chart. Left ventricular systolic dysfunction (LVSD) was defined as any measured value of the ejection fraction equal to or less than 40% documented in the chart from a previous or current hospital admission or by a narrative statement.

For the second quality indicator (angiotensin converting enzyme inhibitor (ACEI) by dose level) we included only patients with LVSD and recorded all ACEI and angiotensin receptor blockers (ARB) prescribed at discharge. We further measured the proportion of patients discharged on ACEI who received the target dose. The target dose was defined as that found to increase survival in patients with LVSD in controlled clinical trials. Target doses were defined as: captopril 50 mg three times daily, enalapril 10 mg twice daily, lisinopril 20 mg once daily, and ramipril 5 mg twice daily.²² If evidence from clinical trials was not available, target dose levels were based on the manufacturers’ stated average doses which were as follows: benazapril 20 mg once daily, fosinopril 20 mg once daily, quinapril 10 mg twice daily, perindopril 4 mg once daily, and cilazapril 1 mg once daily.²³ Finally, we defined three treatment groups: (1) no ACEI at discharge, (2) suboptimal dose of ACEI, and (3) target dose of ACEI or ARB (any dose). Patients with any known contraindications to ACEI were excluded from the analysis.

The third quality indicator was the prescription at discharge of all β blockers for systolic dysfunction. All β blockers prescribed at discharge were recorded, and only patients with LVSD and no contraindication to β blockers were included.

The fourth quality indicator was the prescription of anticoagulants at discharge in patients with atrial fibrillation. Only patients with atrial fibrillation and no contraindication to anticoagulants were included. Atrial fibrillation was recorded on the ECG report during hospitalisation.

Interrater reliability was assessed in one hospital by a random replicate sample of 100 charts which were re-abstracted.²⁴ The kappa values for quality of care measures were 0.91 for the determination of LVF, 1.00 for target dose ACEI versus less than target dose, 0.50 for β blockers in patients with LVSD, and 1.00 for anticoagulants for atrial fibrillation.

Statistical analysis

Bivariate analyses were performed using χ² tests, Fisher’s exact tests, or t tests as appropriate, followed by multivariate analyses. A logistic regression for dichotomous outcomes and an ordinal logistic regression when the dependent variable had three categories (ACEI by dose level) were performed. Selection of the variables included in the model was based on a priori consulting of bivariate results. The fit of all models was good and no collinearity problems were found.^25–27 It was not possible to implement a correlated data analysis taking into account hierarchical structure of the data because there were only three hospitals in the study and this type of analysis requires a minimum of four. To test the validity of readmission we assumed that chart based process indicators represent care actually received and can be used as a gold standard. Sensitivity, specificity, positive predictive value and negative predictive value were measured. Positive and negative likelihood ratios were also computed.^28–30 We tested the validity of readmission by using process quality indicators as gold standard. All analyses were done with SAS software (SAS Institute Inc, Cary, NC, USA).

RESULTS

Of 1634 eligible patients discharged from three Swiss academic medical centres between 1 January and 31 December 1999, we excluded 134 patients who transferred to another acute care facility, six patients who left the hospital against medical advice, and four patients with an uncertain date of discharge. We also excluded 306 patients for one or more of the following conditions: aortic stenosis (n = 111), acute myocardial infarction (n = 107), chronic renal failure on dialysis (n = 31), heart failure attributed to thyrotoxicosis (n = 30), cor pulmonale or chronic obstructive pulmonary disease (COPD) requiring home oxygen (n = 30), mitral stenosis (n = 9), amyloidosis (n = 5), and thiamine deficiency (n = 1). We further excluded 98 who died during their hospital stay.

A total of 1055 patients discharged alive with HF were therefore included in the analysis. The mean (SD) age was 75.0 (12.8) years and 45.7% were women. The distribution of risk factors among these patients was as follows: 56.5% had a previous history of HF, 32.9% prior myocardial infarction, 19.6% COPD, bronchitis or emphysema, 61.1% hypertension, 23.4% diabetes, and 15.4% were current smokers. The mean (SD) minimum ejection fraction was 36.5 (15.6)%, mean (SD) Charlson comorbidity index was 1.91 (1.42), and mean (SD) length of stay was 14.0 (15.0) days.

Predictive ability of outcome indicators

Bivariate analysis

Associations between demographic characteristics, characteristics of patients at admission and when in hospital, and readmissions are shown in tables 2 and 3. Of the 1055 patients, 139 (13.2%) were readmitted within 30 days. Patients with COPD, bronchitis, or emphysema were more likely to be readmitted (18.8%, p = 0.005), as were patients with diabetes mellitus (16.8%, p = 0.048) and current smokers (19.2%, p = 0.010). Patients readmitted had a mean (SD) length of stay of 15.2 (13.7) days which was only slightly longer than the length of stay of patients not readmitted (table 3). Associations between readmissions and discharge characteristics are shown in table 4.

Multivariate analysis

The results of the multivariate analysis are presented in table 5. As covariate we included only variables available in the hospital discharge data. After controlling for hospitals, the risk of LVF not being determined was 0.70 times higher in patients who were readmitted than in patients not readmitted. This relationship was not statistically significant (p = 0.103). Readmissions were also not associated with the three treatment categories of ACEI in patients with LVSD. The adjusted OR (95% CI) from the ordinal logistic regression for the effect of the three categories (no ACEI, less than target ACEI dose, and target dose of ACEI or ARB) was 1.07 (0.56–2.06), p = 0.840. Readmissions were also not associated with the prescription of β blockers in patients with LVSD or of anticoagulants in patients with atrial fibrillation.

Validity and discriminating power of readmissions

The validity and discriminating power of readmissions as predictors of process of care quality indicators are presented in table 6. We tested the validity of readmissions by using process quality indicators as the gold standard. For all the potential associations examined the sensitivity was 0.08–0.17, specificity 0.86–0.88, positive predictive value 0.16–0.43, negative predictive value 0.61–0.80, positive likelihood ratio 0.58–1.24, and negative likelihood ratio 0.96–1.06.

DISCUSSION

In this analysis we found that readmission to hospital was not associated with the quality of care for patients with HF and that it has low specificity and sensitivity. These findings provide new evidence that the use of this outcome as a quality of care indicator is limited, at least when it is measured from routinely collected data—that is, independent of clinical or other events that may indicate that the readmission occurred probably not because of a quality of care problem. One possible explanation for our negative results is that we were not able to measure the intervening variables between process of care and outcome such as changes in disease status and consequent clinical course or other events that may account for the readmission.

Several studies have shown similar patterns.^5,31,32 In a recent study performed in a Swiss university hospital, early readmission by patients with HF was not associated with the quality of care while in hospital, particularly admission work up, evaluation and treatment during the patient’s stay. However, the authors did show that readmission was strongly related to patient characteristics including age, previous diagnosis of HF, and prior myocardial revascularisation.¹¹ Other studies have shown similar relationships between patient characteristics and readmissions.^6,7,10

Despite these results, conflict about the validity of readmission as a quality indicator still remains. Early readmission is sometimes interpreted as a problem following discharge due to inadequate care during the hospital stay, but other factors occurring after discharge may contribute to readmission. Several studies, including a meta-analysis, have shown that readmissions are associated with the quality of care.^1,9,33

Given these conflicting results, the current limitations of routinely available outcome measures such as readmissions and the lack of valid risk adjustment methods, the AHA/ACC Scientific Forum on Quality of Care and Outcome Research in Cardiovascular Disease and Stroke does not recommend the use of readmission for comparing hospitals for patients with HF. However, this outcome measure might be documented and recorded over time and included as part of quality improvement projects within institutions.¹² It is particularly important to understand the limitations of outcome measurements. Despite major advances in the last decade, our ability to adjust outcomes for severity of illness, comorbidity, and other patient related risk factors is still crude and limited.

One limitation of this study is that the available data did not allow us to differentiate between planned and unplanned readmissions, which might have resulted in misclassification bias. Another limitation is the use of process measures as a gold standard to assess the validity of outcome indicators. We know that perfect quality of care is an abstract concept that cannot be exactly measured. The process indicators used were the best available and were therefore designated as “the gold standard”. But these indicators are far from perfect. A further limitation of the study was that the quality of the administrative data was heterogeneous between providers. Specifically, we were able to select only 234 patients with ICD-10 HF codes from one university hospital compared with 976 and 774 patients from the others, creating a potential selection bias.

In conclusion, we found further evidence that early readmission, measured using routinely collected data, is a poor predictor of quality of care indicators for patients with HF. Our findings are consistent with recommendations from the AHA/ACC Scientific Forum on Quality of Care and Outcome Research in Cardiovascular Disease and Stroke to be cautious in using readmissions as quality indicators to compare healthcare providers until the definition and measurement of this variable is improved and new appropriate risk adjustment methods are available.