Among adults hospitalised for common conditions, numerous clinical interventions are established as being effective with benefits that usually outweigh the risks.1^–9 However, these proven interventions are frequently not implemented.7^–13 This may lead to suboptimal patient outcomes, including higher morbidity and mortality, prolonged hospital stay, increased use of clinical resources and decreased quality of life.691415

Some examples of inpatient-based underutilisation contrary to published guidelines and evidence include initiation of osteoporosis treatment in only 3–25% of hospitalised patients with documented fractures,15^–19 initiation of statin therapy in only 6–33% of patients with documented coronary artery disease,1520^–22 aspirin prescription in only 53–68% of patients in whom it was indicated for stroke and/or MI prevention,102223 use of warfarin for stroke prevention in atrial fibrillation in <50% of eligible patients24 and use of angiotensin-converting enzyme inhibitors (ACEIs) in only 50–60% of patients with heart failure.25

Most studies evaluating the appropriateness of therapeutic interventions have focused on the avoidance of iatrogenic adverse effects as a process indicator of quality of care.26^–31 Recently, increased attention has been given to the evaluation of therapeutic omissions as another critical element of quality of care.10^–123233 A recent Canadian report revealed that 57% of in-hospital adverse events resulting in death, physical impairment or disability among medical inpatients events were due to errors of diagnostic or treatment omission.9 Therapies omitted in hospital may also significantly impact posthospitalisation adverse event rates.614

Underutilisation represents missed opportunities to optimise outcomes and quality of care based on the best available evidence. Most studies have focused on the implementation of one or two interventions.113334 Although these results are helpful in identifying specific areas of deficiency, by incorporating a wider range of conditions we can assess the quality of care in more areas of practice but also assess whether there are barriers to implementation affecting care overall. Furthermore, patient-specific reasons for non-use such as contraindications, history of adverse effects and patient preferences have rarely been previously considered.35^–38 Some studies have used International Classification of Diseases (ICD) codes, surveys and prescription or drug claim databases to identify patients and implementation outcomes without reviewing patient health records to identify reasons as part of their methodology.1217^–193940 Such studies likely underestimate the quality of care being provided by disregarding instances where not implementing an intervention is the most appropriate action.

Hence, to our knowledge, there are no published reports describing the overall implementation rate of proven interventions among medical inpatients that account for valid reasons for non-use of the interventions.

Objectives

From the perspectives of quality of care and patient safety, we sought to determine the overall implementation rate of a broad range of proven interventions among adult general medicine patients while taking into account valid reasons for non-use. Secondarily, we sought to evaluate the feasibility of generating numerical indices to describe the implementation rate of such interventions, and to identify predictors of implementation.

METHODS

This was a retrospective observational study. Based on our knowledge of the conditions commonly affecting the patients in our area of clinical practice, we chose to evaluate the following conditions: atrial fibrillation, coronary artery disease, ischaemic stroke, systolic heart failure, type 2 diabetes, and osteoporosis. We reviewed current practice guidelines and current clinical trials related to those conditions and identified 17 proven interventions (15 pharmacological and two non-pharmacologic) for evaluation (table 1). We defined “proven interventions” as those that were generally accepted as being effective with benefits that generally outweigh their risks and were supported by high-quality evidence and/or reputable practice guidelines. Valid reasons for non-use of each intervention were based on published contraindications and common adverse effects of the pharmacological interventions (table 2).

Subjects

Patients discharged between August 2005 and April 2006 from the general medicine wards at an urban community hospital were identified through a list generated by a data analyst. The hospital is a 90-bed institution staffed by an interdisciplinary team of hospitalists and internists with consultant specialist support, clinical pharmacists, dieticians, nurses and rehabilitation therapists. Potential subjects were identified by our institution’s Quality Utilization Information Support Team based on discharge reports from target medical units, stratified by month and screened for study inclusion. Any patient 18 years of age or older with one or more of the study medical conditions was eligible, but patients were excluded if they were older than 90 years, had documented severe dementia or were receiving palliative or comfort care only. The exclusion criteria were designed to exclude patients in whom it is not generally accepted that the benefits of the study interventions outweighed their risks or in whom the interventions are not generally recommended. We also excluded patients who had a hospital stay <48 h to ensure that physicians, pharmacists and/or nurses attending to the patient during that admission had an appropriate amount of time available for patient assessment. A sample size of 150 was chosen based on available resources and the time frame over which the study had to occur.

Measurements

The primary data sources were individual patient health records which included computerised and paper charts. Records from the incident and previous admissions to the study centre were reviewed to form as complete an assessment as possible.

If an intervention was indicated, we determined whether the intervention was implemented. An intervention was considered implemented if the patient was receiving it during the last 24 h of their hospitalisation or was discharged from hospital with a prescription for the intervention. For non-pharmacological interventions, the implementation criterion was documentation in the health record of the intervention having been performed. Physicians, pharmacists, dieticians or nurses may have implemented interventions. If the intervention was not implemented, the researcher reassessed the records to identify prespecified valid reasons for non-use that may or may not have been explicitly documented. Where no valid reason for non-use was found, a second researcher re-examined the records to ensure reasons for non-use were not missed. If an intervention was implemented but discontinued and not prescribed at discharge, it was not considered to be implemented. However, if a valid reason for non-use was present, it would be categorised as an incident of justified non-use.

Endpoints

We developed two performance indices to describe implementation rates of proven interventions: the Ideal Intervention Index (3I) and Justified Non-Use Index (JNUI). The 3I describes the proportion of ideal intervention opportunities that resulted in an intervention being implemented. We defined an ideal intervention opportunity as an instance where a patient had an indication for a study intervention and no documented or valid reason for not implementing it (table 3).

The Justified Non-Use Index (JNUI) describes the proportion of interventions not implemented that had an identifiable reason for non-use, either through direct documentation of such a reason or by the researchers’ use of prespecified criteria.

These indices were based on the patient safety concept that there should be no unexplained variance from “best practices” in health care.41^–43 Hence, values of 1.0 for both indices were considered achievable. The indices were calculated for the composite of all the interventions, for each medical condition, intervention, and patient. Calculation of the 3I and JNUI on an individual patient basis allowed us to generate 95% confidence intervals for both indices.

Patient characteristics, determined a priori, were analysed for associations with the patients’ individual indices using univariate regression analyses. Linear regression was used for associations with age, length of stay, severity of illness (Cumulative Illness Rating Scale for Geriatrics Severity Index, CIRS-G Severity Index) and number of intervention opportunities per patient. Ordinal regression was used for associations with functional status (Karnofsky Performance Score). A test for association between individuals’ 3I and JNUI and reason for hospital admission was included as an indicator of the validity of the performance indices. Presumably, more study intervention activity should occur when the hospitalisation is due to one of the study conditions. Independent t tests or one-way ANOVA tests were used to evaluate associations with gender, posthospitalisation disposition, reason for admission and stay at a tertiary hospital immediately prior to admission. The descriptive and inferential statistics were performed using SPSS 11.44

RESULTS

We randomly screened 249 patients in order to obtain the target sample of 150 patients. Absence of a study medical condition (n = 45), age >90 years (n = 24) and palliative status (n = 20) were the most frequent reasons for exclusion. The baseline characteristics of the 150 included patients are shown in table 4.

Among 557 potential interventions, 448 ideal intervention opportunities were identified. In 330 of these, an intervention was implemented, yielding an Ideal Intervention Index of 0.74 (95% CI 0.70 to 0.78) (fig 1). Of the 227 interventions not implemented, valid reasons for non-use were found in 109 cases, resulting in a Justified Non-Use Index of 0.48 (95% CI 0.41 to 0.55). There were an average of 3.7 ideal intervention opportunities per patient (table 5).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Overall Ideal Intervention Index and Justified Non-Use Index.

Most (74%) of the interventions implemented were continuations of therapy initiated prior to the hospitalisation. Failure to continue preadmission therapy without a valid reason (reconciliation failure) represented nine (8%) of the 118 ideal intervention opportunities not acted upon.

Forty-six per cent of the reasons for non-use identified were explicitly documented. The other 54% were determined by inference based on transcriptions from other admissions, laboratory values and prespecified intervention-specific criteria.

No statistically significant associations between patient characteristics and their individual 3I or JNUI were found for any of the parameters tests, with one exception: when a patient’s reason for admission was related to one of the patient’s study medical conditions, their individual 3I and JNUI tended to be higher, (0.80 vs 0.62; 95% CI 0.06 to 0.30; p = 0.003 and 0.56 vs 0.38; 95% CI 0.01 to 0.34; p = 0.039, respectively).

DISCUSSION

To our knowledge, this is the first study to assess the implementation rate of proven interventions in such a broad spectrum of common chronic conditions among general medical inpatients.

Proven interventions appeared to be underutilised in the studied population, even after accounting for valid reasons for non-use. There also appeared to be a high rate of unexplained variance in the use of the studied interventions, with only around 50% of the non-use being explained by direct or inferential evidence available in the health records.

Both indices reached 1.00 for the implementation of aspirin in ischaemic stroke, reflecting complete implementation of this intervention. In contrast, smoking cessation therapy/counselling had the lowest indices (3I 0.38, JNUI 0.00). In comparison, approximately 75% of patients with a history of myocardial infarction or heart failure received smoking-cessation therapy or counselling in US hospitals in 2005.45 Brief advice with or without information on the harmful effects of smoking by a physician increases the odds of quitting (OR 1.74, 95% CI 1.48 to 2.05).46

Our study confirms the feasibility of measuring rates of intervention using a relatively simple methodology that involves only health records. Moreover, we believe that the indices, and the 3I in particular, are valid indicators of quality of care. We base this on the achieved 3I of 1.00 for aspirin use following stroke, confirming that it is possible to achieve complete implementation of a proven intervention in all appropriate patients and the significant correlation between patients’ individual 3I and being admitted for a study condition, which lends internal validity to the 3I as a reflection of intervention activity. From a treatment viewpoint, the conditions and interventions we studied are consistent with the “tracer” approach to quality measurement,47 suggesting that the quality of care for the common chronic conditions we studied may be indicative of the quality of other aspects of these patients’ care. This deserves further study. We also believe that our method is robust in light of established criteria for evaluating the quality and validity of a care process measure such as ours.48

Several limitations of our methodology deserve discussion. Obviously, our reliance on the health record predisposes our results to biases resulting from underdocumentation of potentially relevant facts. The low JNUI is particularly susceptible to underdocumentation of reasons for non-use of proven interventions. There are also probably reasons for non-use of the studied interventions that are difficult to anticipate using the objective criteria inherent in our methodology. Some of these may include values-based decisions made by prescribers based on patients’ perceived prognosis, “pill burden,” or health status, and prescribers’ own biases about the individual interventions (eg, cost, risks, inconvenience) or their perceptions about their role in the chronic care of the patient. For example, one study found that 48% of hospital physicians thought patients would prefer that long-term preventive interventions be started by the primary care physician.49 These issues deserve further study. Because we were primarily interested in identifying “sins of omission,” we did not attempt to identify situations where intervention occurred when there were good reasons not to intervene. Future research could attempt to combine the “errors of omission” perspective studied here with the “errors of commission” perspective that the medication appropriateness literature reflects.26^–31 Nonetheless, we perceive the 3I, and to a lesser degree, the JNUI to be reasonable indicators of the quality of pharmacotherapy provided to the studied population during their inpatient stay.

Several further research questions arise from our results: Which other proven interventions are feasible to assess? Can this methodology be transposed to other clinical areas with different sets of proven interventions? Do higher values for 3I and/or JNUI correlate with better health outcomes? What efforts to increase the utilisation of proven interventions are effective and efficient? Some of these questions have been addressed in other patient populations,50 or with limited sets of interventions33495152 We have begun an expanded implementation of our methodology across six acute-care hospitals using a 10-intervention subset of that studied here.

Overall, proven interventions are underutilised in the general medical inpatients studied, and reasons for non-use of the studied interventions are frequently not discernible from the health record despite using objective and inferential criteria to detect them. There is potential for improvement, but research is required to further investigate reasons for non-use. Using the 3I and JNUI indices to measure intervention rates and to reflect the quality of care being provided is feasible.