The translation of research findings into practice is notoriously slow. Several reports have documented the long lag time between publication of clinical trials documenting efficacy of new treatments and the eventual adoption of those treatments in routine clinical practice.¹ To accelerate this process it is often necessary to undertake an active implementation program.¹ Recently, two major clinical trials have shown that treatment with either gemfibrozil or simvastatin reduces major events in patients with ischemic heart disease (IHD) whose primary lipid abnormality is low levels of high density lipoprotein (HDL) cholesterol.^2,3 In both these studies, active treatment was associated with 20–30% relative risk reductions in myocardial infarctions, coronary heart disease death, strokes, and other important clinical outcomes. However, when we examined clinical practice in a group of US Department of Veterans Affairs medical centers, we found that fewer than 20% of eligible patients had been prescribed lipid modifying therapy consistent with this new evidence. We therefore undertook this study to develop and test an intervention to improve prescription of evidence based lipid therapy to patients with IHD and low levels of HDL cholesterol.

A review of the literature suggested that an intervention intended to change provider prescription behavior would be most effective if it was designed based on insights derived from theoretical models of practitioner prescribing behavior; empirical studies of specific interventions used to change such behavior; and data about perceived barriers.^4,5 The latter we obtained from structured qualitative interviews conducted with 14 primary care physicians before the study. Our intervention consisted of education, opinion leader influence, and prompts (fig 1). We used this multi-pronged approach because interventions consisting of multiple components have been shown to be superior to single component interventions.⁶ Educational workshops were intended to provide educational content, to increase the degree to which providers have a positive attitude toward prescribing lipid modifying medication for the target population, and to create normative support. Opinion leaders were used to increase perceptions of professional acceptability and subjective norms regarding prescribing for these patients. Prompts were used to increase the likelihood that providers would remember to consider and implement the new behavior in spite of the environmental barriers created by competing demands and distractions during encounters. Although prompts have been found to be an effective strategy for increasing provider compliance with preventive measures,^7,8,9,10 only a few studies have directly compared the efficacy of different types of prompts.¹¹

Our primary objective was to evaluate the effectiveness of this multicomponent intervention for improving evidence based lipid management for IHD patients with low HDL cholesterol. A second objective was to evaluate the relative effectiveness of three different prompt types on provider prescribing behavior.

METHODS

Overview of study design

The study was conducted from 1 October 2001 to 1 October 2003; the intervention was implemented beginning in April 2002 and continued for approximately 1 year. The effect of the intervention on the proportion of eligible patients receiving lipid modifying therapy (objective 1) was compared between five intervention sites and six matched control sites using a controlled before and after study design.¹² This study design allowed us to evaluate the effect of the intervention controlling for secular trends. To determine the relative effectiveness of different types of prompts (objective 2), we randomized providers within the intervention clinics (using a balanced block randomization scheme with blocking of providers by case load) to progress notes, patient letters, or computer chart reminders.

Intervention components

Educational approaches (written materials and educational workshop)

The development of educational materials was guided by marketing and persuasion theories^13,14 as well as Maibach and Cottone’s¹⁵ social cognitive approach to message design. A 1 hour interactive educational workshop tailored to each site and facilitated by a physician co-investigator was presented at each intervention site to introduce primary care providers to the study and to educate them as to appropriate treatments for patients with low HDL cholesterol and IHD. The physician co-investigator was introduced by the local opinion leader, lending credibility to his message. The workshop included a Power Point presentation reviewing the published evidence regarding use of medications for primary and secondary prevention of heart disease. This usually took about 40 minutes. Following the standardized presentation, the discussant facilitated a discussion among providers about how they interpret the evidence and use it in making their own practice decisions. Lunch and continuing medical education credit were provided to workshop participants. Providers who could not attend the workshop were sent the written materials.

Opinion leader influence

All primary care providers within the intervention sites were surveyed in order to determine which colleagues they felt were most influential in their practice of medicine. Opinion leaders at each site were selected based on their number of mentions on this survey. The opinion leaders were initially contacted by telephone by the principal investigator several weeks before the educational workshop at each site. Thereafter, the study coordinator contacted the site opinion leaders on a bimonthly basis. The role of the opinion leaders in the study was to encourage providers to increase their prescription of lipid therapy for the target patient population.

Prompts

We tested the relative effectiveness of three different types of prompts: patient letters, computer chart reminders, and progress notes. Within each clinic, providers were randomized to one of these three. Prompts were generated only for patients in the study cohort who were not receiving a lipid medication, based on VA administrative records.

• Patient letters were sent 1–2 weeks before the patient’s visit to his primary care clinic. A cover letter with a brightly colored card gave information about lipid management for IHD patients and suggested that the patient should bring the letter and discuss its contents with his provider at the beginning of his forthcoming visit. The emphasis on an early mention was based on findings that requests for medication that occur early in the encounter are more likely to result in positive outcomes.¹⁶ These materials may be viewed at http://www.hsrd.minneapolis.med.va.gov/PageResearchTools.asp.

• Chart reminders consisted of a short phrase that appeared on the cover page of the patient’s computerized medical record in a box entitled “reminders due”. This reminder was visible to the clinician when s/he opened a specific patient chart at the time of the visit. Reminders were written so that they would not reappear once a lipid lowering prescription had been written.

• Progress notes consisted of a short message to the primary care clinician reminding him/her about the appropriate approach to the lipid management of the IHD patient with the target lipid profile. When the clinician logged onto the computerized patient record system s/he encountered a screen that included a “notifications” box. In this box was a message stating that there was a progress note requiring co-signature on a particular patient. When the provider selected that notification, the progress note appeared. The progress note was sent to providers randomized to this arm a few days before the target patient’s visit.

Study sites

Primary care clinics at each of five Department of Veterans Affairs medical centers (box 1) in the Upper Midwest of the United States served as intervention sites. Control sites consisted of six other facilities matched to the intervention sites by facility characteristics (size and academic affiliation, table 1). The sample consisted of 92 primary care providers (physicians, nurse practitioners, physician assistants) who agreed to participate in the study. The study was terminated 2 months early in one intervention site due to cessation of all human subjects research there, an event unrelated to this study.

Patient cohorts

Target patients were identified based on the following:

• a diagnosis of IHD during the 5 years before the assessment period;

• regular follow up in one of the target clinics (defined as three or more visits within the past 2 years);

• most recent low density lipoprotein (LDL) cholesterol <130 mg/dl and HDL cholesterol <40 mg/dl; and

• no lipid modifying medication prescription during the 6 months before the start of the study assessment period.

To identify patients with IHD we used a previously validated algorithm based on inpatient and/or outpatient diagnoses, surgical procedures (coronary revascularization procedures), and outpatient prescriptions for anti-angina medication.¹⁷

Outcomes

The primary outcome was the proportion of target patients prescribed any lipid medication—that is, fibrates (such as gemfibrozil), statins, bile acid binding resins, or niacin.

To evaluate the prescribing trends in the year before the intervention, we identified all patients at both intervention and control sites who met the target criteria at the start of the pre-intervention period, 1 year before the start of the intervention. The proportion of this cohort who were prescribed any lipid medication at the end of the year was then assessed. To evaluate the prescribing trends in the year during the intervention we identified all patients at both intervention and control sites who met the target criteria at the intervention start date. The proportion of this cohort prescribed any lipid medication 1 year later was then assessed.

Statistical analysis

Objective 1

A hierarchical or random effects logistic regression analysis modeled the log odds of a prescription as a function of the intervention group, evaluation period, and an interaction between period and intervention group. The model incorporated random effects for site within intervention group and random effects for an interaction between period and site within intervention group to formally model correlations between outcomes within sites and to allow for variation across sites in the effect of the interventions. The effect of particular interest, evaluated with a likelihood ratio test, was the interaction between intervention group and evaluation period. The intervention was expected to lead to an increase in prescriptions from the pre-intervention period to the intervention period in the intervention sites, while the prescription rate was expected to remain stable across periods in the control sites.

Objective 2

A hierarchical logistic regression analysis modeled the log odds of a prescription as a function of the different prompts used in the intervention site, and an interaction between site and intervention prompt together with random effects for provider within site. An indicator for whether any prescription had been given in the months subsequent to cohort identification yet before the start of notifications was also included in the model as a covariate. Likelihood ratio tests were used to examine whether the prompts resulted in different rates of prescription and whether the effects of the prompts varied across sites. Examination of the rates of prescription for the three prompts suggested an aggregation of sites into three distinct groups according to how the progress note compared with the reminder. An alternative analysis modeled the log odds of a prescription as a function of the different prompts, site, and an interaction between prompt and aggregated site group, together with random effects for provider within the site. In order to determine whether workshop attendance was associated with prescription rates, we performed both a stratified analysis and incorporated workshop attendance into the model. All analyses were performed with SAS software, version 8.02 (SAS, Cary, NC, USA).

Sample size

The target sample size of 11 700 patients (including pre- and post-intervention cohorts at both intervention and control sites) was based on prevalence estimates from the Upper Midwest Network, and was calculated to provide 80% power at a significance level of 0.05 to detect differences between the post-intervention concordant treatment rates for the intervention and control sites and for any two prompt systems when the actual differences were as small as 10% and 7%, allowing for moderate intraclass correlations within sites and physicians.

Process evaluation

Process evaluation data were collected through a self-administered survey for providers administered before and immediately after the educational workshop and then at the conclusion of the intervention period. The surveys were field pre-tested using standard survey research methods¹⁸ and revised as needed. Simple summary statistics for provider attitudes, perceived norms, and behavioral intentions as measured on the surveys were constructed.

The study was approved by the Institutional Review Board of each participating facility.

RESULTS

We identified 111 primary care providers at the five intervention sites: 60 (54%) attended the workshop and 92 (83%) agreed to participate and were randomized to the three prompt groups (fig 2). A total of 9105 patients comprised the four cohorts (table 2). The mean (SD) age was 71 (10) years; 99% were male; 6% were known to be non-white; and 66% were currently married. Mean (SD) LDL cholesterol was 94 (22) mg/dl and mean (SD) HDL cholesterol was 32 (5) mg/dl.

Objective 1

The prescription rate at the control sites did not change between the two time periods (pre-intervention period 18.9%, intervention period, 17.7%, p = 0.19). In contrast, the prescription rate increased from 8.3% during the pre-intervention period to 39.1% during the intervention (OR = 6.5, 95% CI 5.2 to 8.2, p<0.0001) at the intervention sites. The logistic regression analysis indicated that the interaction between group (control v intervention) and time period was highly significant (p<0.0001). The adjusted odds of receiving a prescription during the intervention period was 3.1 times higher at the intervention sites than at the control sites (95% CI 2.1 to 4.7). Results for each intervention site were consistent with the overall group findings. Attendance at the workshop was not associated with prescription rates.

Objective 2

Overall, there was no statistically significant difference in prescription rates among the three prompt groups (40.7% for progress notes, 36.9% for patient letters, and 39.4% for reminders, p = 0.60). However, in the alternative logistic regression analysis there was a significant interaction between group and site, indicating that the efficacy of the prompts differed by site. Specifically, at Fargo, North Dakota, all prompts appeared to work equally well. At Minneapolis, Minnesota and Sioux Falls, South Dakota, progress notes were significantly more efficacious than reminders (p = 0.01). In contrast, at Black Hills, South Dakota and St Cloud, Minnesota, reminders were significantly more efficacious than progress notes (p = 0.04). The relative effectiveness of patient letters presented no simple summary pattern but was roughly equivalent to other types of prompts when site results were combined.

Process evaluation

The response rates to the surveys are shown in fig 2. The process evaluation was intended both to check the actual exposure to the intervention and to describe the experience of those exposed.¹⁹ The main findings were:

• Providers thought the educational workshop was helpful. Specifically, 99% of respondents agreed somewhat or completely that the workshop increased their knowledge about how to treat IHD patients with low HDL cholesterol and 86% agreed somewhat or completely that they were more likely to treat these patients with medication as a result of the workshop.

• The opinion leader talked to the providers about management of low HDL cholesterol in IHD patients. 61% of respondents agreed that the opinion leader had spoken to them on at least one occasion about lipid management for low HDL cholesterol and 60% agreed or strongly agreed that the conversations with this provider or other colleagues influenced their prescribing decisions.

• Reaction to the prompts was mixed. Only about half the providers agreed or strongly agreed that prompts positively influenced their lipid management and 40% found the prompts annoying. 14% believed that “prompts of this kind do more harm than good”.

DISCUSSION

This study showed that an intervention that included educational workshops, opinion leader influence, and prompts substantially influenced provider prescription behavior resulting in a significant increase in the proportion of IHD patients receiving evidence based lipid therapy. The 32% effect size noted in this study (an increase in prescription rate from 8% to 40%) was consistent across all sites in the intervention group and was considerably larger than that reported in the literature where the average effect size for prompt based interventions is about 13%.^6,10

We attribute this success to the tailored development of a targeted intervention that was based on (1) a theoretical framework that integrated “persuasive communication, diffusion of innovation, social influences, adult learning theory and social cognition”;⁴ (2) the empirical literature on the efficacy of modalities to change provider behavior; and (3) careful identification of specific local barriers.⁵ Other strengths of this study were the large sample size, the inclusion of a process evaluation, the randomized comparison of the three prompt types, and the use of analytical techniques that accounted for the clustered nature of the data.

Our study did not unequivocally demonstrate the superiority of any single prompt type across all sites, possibly due to inadequate power to detect interaction effects resulting from variation across sites in the relative effectiveness of prompts. In one site, all three prompts worked equally well while in two sites reminders were more efficacious and in two others progress notes were more efficacious than the other prompts. Unfortunately, the process evaluation was unable to shed much light on this hetereogeneity.¹⁹ While it is always possible that chance is responsible, we speculate that these results may reflect the different cultures prevalent in the intervention site primary care clinics. For example, in some sites reminders are accepted by clinicians as useful clinical tools whereas in others they are viewed with suspicion as administrative tools used to drive performance and control costs. Indeed, the rationale behind testing the relative efficacy of progress notes, a unique type of prompt, was that these would be perceived as less threatening and more palatable to clinicians than formal reminders which are associated with the spectre of performance measurement and, among some providers, administrative interference in clinical decision making. Our finding that progress notes were more efficacious than reminders in some sites indicates that, at least in some clinical settings, separating performance measurement from reminder functions might be beneficial.

Previous studies have shown that reminders are generally more effective than other interventions such as educational materials, audit and feedback, or usual care, but there are few comparative data on the relative efficacy of different types of reminders.¹¹ We could only identify one such study, an uncontrolled before-after study.²⁰ In this study reminders to providers were more efficacious than patient letters. Even as single interventions, patient mediated interventions have not been well studied; a recent review identified only seven studies that compared patient mediated interventions against a no-intervention control group. This review concluded that, although all studies were plagued by unit of analysis errors, these interventions might result in “moderate to large improvements in performance, especially when targeting preventive services”.¹¹ It has been suggested that patient mediated interventions might be particularly efficacious when the primary responsibility for the desired action lies with the patient, and that provider prompts are more efficacious when it lies with the physician.

The major limitation of this study is that, for the primary objective, we did not use a randomized controlled trial design. Nevertheless, controlled before-after study designs are widely regarded as acceptable for implementation studies and our study was conducted in accordance with quality criteria advocated for this type of study.^11,12 A common problem with this study design, which we did encounter here, is heterogeneity of baseline performance between control and intervention sites. During the pre-intervention phase the control sites had a higher rate of lipid therapy in the target population than the intervention sites. We speculate that this may be due to differences in patient populations. Patients in our intervention sites were, on average, older and sicker (more co-morbidities and more hospital admissions, data not shown) than those in control sites, characteristics that may have led providers to be less inclined to prescribe lipid medications. However, even after adjusting for these known baseline differences in multivariate modeling, the results did not change. It is also possible that this difference in baseline rates is due to unknown differences among provider groups, although this seems less likely. It should be noted that providers in the intervention sites did not know that their sites had low pre-intervention prescription rates. Another limitation of the study is that, as for all multicomponent intervention studies, it is difficult to determine which component(s) actually influenced the outcome. Our process evaluation suggested that providers perceived the educational component and opinion leader influence more favourably than the prompts. Finally, we do not have any information on the attitude or behavior of the patients nor do we have sufficient follow up data to know whether the effect of the intervention was sustainable over time.

In conclusion, this study shows that a relatively simple intervention (an educational workshop, opinion leader influence and prompts) based on a theoretical model of provider behavior which is designed to address empirically identified barriers can result in substantial improvement in provider prescription behavior. It also underscores the importance of tailoring quality improvement interventions to the unique structural and cultural characteristics of each healthcare site.