BACKGROUND

The present study is part of a series of studies which evaluated a randomised controlled trial on the quality of cardiovascular and diabetes care in general practice.^11–18 A total of 124 practices with 185 GPs participated in this trial. Baseline assessments provided insights into the actual quality of care.^11–14 The trial also showed the feasibility, effectiveness, and costs of a multifaceted intervention to improve certain elements of the structure, process, and outcome of care.^15–18 The intervention comprised feedback reports and outreach visits from trained non-physicians. The findings of the different studies are summarised in table 1. The focus of the present study is thus on the effect of the intervention on various aspects of the process of care.

METHODS

The participants, intervention, assessments, randomisation, and statistical analysis have been described elsewhere¹⁷ and are reproduced here.

Participants

Practices in the southern part of The Netherlands were recruited via bulletins and by letter until a total of 124 practices were randomised. The recruitment lasted from May 1996 until May 1997. The inclusion criteria were the presence of a clinical computer system, employment of one or more practice assistants, and no major changes in personnel or premises planned during the course of the trial. We considered these criteria crucial for the conduct of the improvement project. Our offer included a certificate for six hours of accredited training for each participating GP (a GP in The Netherlands has to collect 40 hours of accredited training per year), and for the control group also feedback reports and 225 Euro per practice. The feedback reports for the control group were based on post-intervention measurement, supported by a manual to improve the general practice management of patients at high cardiovascular risk, and were sent after the trial. The first practices started their baseline measurement in November 1996 and the last practices finished their post-intervention measurement in July 1999.

Intervention

The GPs in the intervention practices received feedback reports and support from facilitators to improve clinical decision making for patients with hypertension, hypercholesterolaemia, diabetes, angina pectoris, heart failure, transient ischaemic attacks, or peripheral arterial disease. The intervention comprised seven outreach visits per practice (one outreach visit per medical condition). Each practice received support from one facilitator; in partnerships, the facilitator could see more than one GP at the same time. Before each visit all GPs in the practice received a feedback report on the medical condition to be addressed during the visit. The feedback reports were based on baseline performance data and informed the GPs about their current clinical decision making in relation to the key recommendations from the national guidelines issued by the Dutch College of General Practitioners (DCGP).^19–25 During the visit the facilitator and the GPs discussed the content of the feedback reports, prioritised specific aspects of decision making for improvement, and made change plans. The facilitator provided guidance, support, and educational materials to achieve improvement (box 1).

The intervention was part of a larger implementation project concerned with both practice organisation and clinical decision making with regard to patients at high cardiovascular risk. The focus of the project was on the implementation of a comprehensive programme of recommendations derived from the DCGP guidelines and consensus procedures. The facilitators conducted 15 outreach visits per practice, lasting an average of one hour per visit, equally distributed across a period of 21 months. The first eight visits concerned practice organisation; the other seven visits concerned clinical decision making. The protocol for the visits was highly standardised to limit variation and based on a model of change.⁸ The facilitators were specially trained to carry out the project protocol and to support the GPs. The training comprised lectures by the researchers (80 hours including 25 hours on clinical decision making for patients at high cardiovascular risk) and outreach visits in one pilot practice per facilitator. Each facilitator was supervised by one of the GP researchers during the entire intervention period. Most of the facilitators had worked as practice assistants in the past but none were trained physicians. In The Netherlands, practice assistants are qualified to perform administrative and organisational tasks (including the triage of patients) as well as medical activities such as blood pressure measurement and the provision of lifestyle advice.

Outcome measures

The outcome measures were the compliance rates for evidence-based indicators for the actual management of patients at high cardiovascular risk. We used the DCGP guidelines to identify the indicators. These guidelines are based on scientific evidence, broad consensus, and clinical experience.²⁶ A group of five GPs, involved in research and guideline development but not participating in the trial, were asked to select the key recommendations from the guidelines. The recommendations are detailed descriptions of specific clinical actions together with the clinical situations calling for those actions. The selected key recommendations coincide with the recommendations from international guidelines.^1,27–29 The key recommendations were grouped into indicators pertaining to specific aspects of cardiovascular care (see example in box 2). The outcome measures therefore indicated quality of clinical decision making and not just performance of clinical actions because the indicators also included the clinical situations calling for those actions.

After the trial we selected all the indicators which allowed detection of an absolute difference of 15% in the compliance rates between intervention and control groups with more than 90% power at a 5% level of significance. Changes in care provision are usually no more than 10%.³⁰ None of the indicators concerning clinical decision making for transient ischaemic attacks or peripheral arterial disease met the selection criteria and the results for the indicators for diabetes are presented elsewhere.¹⁷ The 12 selected indicators are presented in table 3. The smallest difference we would have been able to detect across the indicators was 7% (α=0.05, β=0.10). The post hoc power estimations take into account the design effect of cluster randomisation.³¹

Assessments

Using the key recommendations, forms were developed for the prospective recording of patient encounters for each of the medical conditions considered in the trial. Because of the high incidence of hypertension, separate forms were created for newly diagnosed and already treated (with or without medication) hypertension. The encounter forms included items pertaining to the age, sex, and clinical characteristics of the patient, and also the decisions regarding the performance (yes/no) of specific clinical actions. While the forms were based on the key recommendations, they did not contain any clues to the recommendations. GPs have been shown to complete similar forms reliably (kappa=0.79; Spies TH, personal communication).

The GPs completed encounter forms during routine consultation hours at baseline and post-intervention measurement for a period of 2 months. They were asked to complete the forms immediately after eligible encounters. The data from the encounter forms were then entered into a computer by personnel blind to group allocation.

The characteristics of the participating practices were derived from a questionnaire completed by one GP per practice at baseline. Data were collected on type of practice (single handed versus partnership), practice location, number of GPs and practice assistants, working hours of each professional, age of GPs, patient list size, and involvement in GP vocational training.

Costs

The costs of the 21 month intervention were calculated using data provided by the facilitators and salary scales. The calculations included the time which the facilitators spent preparing and making the visits, their travel costs, and also the time spent by GPs to attend the visits. Moreover, the amount of time the GPs spent reading the feedback reports and carrying out the change plans was obtained by the facilitators and included in the calculations. The calculations did not include the costs for generating the feedback reports and training the facilitators because, per practice, these costs are strongly influenced by the number of participating practices.

The costs of the intervention for clinical decision making were estimated for the four medical conditions studied—that is, hypertension, hypercholesterolaemia, angina pectoris, and heart failure. An exact calculation was not possible because organisational arrangements and the implementation of guidelines for diabetes, transient ischaemic attacks, and peripheral arterial disease may also have influenced the clinical decision making studied. We estimated the costs of the intervention reported in this paper at 40% of the calculated costs for the entire 21 month intervention.

Randomisation

Immediately after the baseline measurement each practice was randomly allocated to receive intensive support (intervention group) or no special attention (control group). The practices were numbered and the person responsible for the randomisation process was blind to the practice identities. A random number generator was used to select permuted blocks with a block size of four. Practices were stratified according to practice type (single handed versus partnership) as this characteristic has been found to predict change in practice organisation.⁸

Statistical analysis

The practice was the unit of analysis for describing changes in clinical decision making. Data from the encounter forms were used to calculate the mean compliance rate for each indicator at baseline and the mean change from baseline. The compliance rate for an indicator was the number of decisions in accordance with the recommendation(s) of that indicator divided by the total number of decisions made with respect to that indicator. The calculations pertained mainly to clinical actions performed during the patient encounters included by the GPs. Particular actions were, however, also taken to be performed when the GP reported performance in a previous contact (such as advice on smoking cessation) or within the recommended period (such as blood pressure measurement within the last 12 months). Actions with missing data were considered not performed. As already mentioned, the calculations took detailed account of the clinical situations.

Multilevel logistic regression analysis (GLMMIX procedure in SAS) was used to assess the influence of the intervention on clinical decision making.³¹ Multilevel analysis takes into account the relatedness of the clinical decisions made within a particular practice. We also used these analyses to identify practice characteristics which were related to success of the intervention and to determine the influence of the baseline compliance rates of the different practices on the effect of the intervention. p values of <0.05 were considered statistically significant.

RESULTS

A total of 157 practices were assessed for eligibility and 33 (21%) were excluded (fig 1). Table 2 shows the baseline characteristics of the 124 participating practices (185 GPs) and the number of patient encounters. Four intervention practices (4/62 = 6%) did not receive a per protocol intervention. These practices received feedback but no support from a facilitator with regard to clinical decision making. Three other practices were lost to follow up, so 121 practices (98%) completed the trial (fig 1). In general, the age and sex of the patients were found to be evenly distributed between the intervention and control groups for the encounters at baseline and post-intervention measurement. The proportion of men with treated hypertension at post-intervention, however, was lower for the intervention group than the control group (39.2% v 46.0%, p=0.003, multilevel analysis). The GPs reported 30 101 clinical decisions at baseline and 22 454 decisions post-intervention (table 3).

Outcome measures

The intervention resulted in statistically significant improvement for five of the 12 indicators (intention to treat analyses, table 4): assessment of risk factors in patients with hypercholesterolaemia or angina pectoris, provision of information and advice to patients with hypercholesterolaemia or treated hypertension, and checking for clinical signs of deterioration in patients with heart failure. Exclusion of those practices which did not provide data for a particular indicator either before or after the intervention had marginal effects on the findings for that indicator. Single handed practices, non-training practices, and practices with older GPs benefited most from intensive support on the provision of information and advice to patients with hypercholesterolaemia or treated hypertension. Other practice characteristics were found to be associated with effects for only one or no indicators (table 5). The baseline compliance rates of the different practices did not influence the effect of the intervention.

Costs

Our estimate of the average costs of the intervention for clinical decision making for the four medical conditions studied in this paper was 1500 Euros per practice. This included an average of 11 hours spent per GP in the intervention group.

DISCUSSION

Intensive support from facilitators improved the clinical decision making of GPs with regard to the assessment of risk factors and the provision of information and advice for certain types of patients at high cardiovascular risk, and also the checking for clinical signs of deterioration in patients with heart failure. Single handed practices, non-training practices, and practices with older GPs gained particular benefit from the intervention. The effects on the clinical decision making were, however, small and the average cost per practice was 1500 Euros.

The intervention may also ultimately improve patient outcomes. Identification of the presence of risk factors provides opportunities to reduce risk, whereas early detection of deterioration in patients with heart failure may promote adjustment of treatment and thereby prevent hospitalisation.³² Furthermore, the provision of information by doctors has been found to improve compliance with treatment, satisfaction with the care delivered, and health outcomes, whereas lifestyle interventions aimed at patients at high cardiovascular risk have been shown to reduce morbidity and mortality.^33–36

Just why the intensive support was effective for five of the 12 indicators remains unclear. Comparison across the indicators is hindered by differences in the baseline compliance rates and power of the study. Moreover, we do not have insight into the motives of the GPs for apparently ignoring certain recommendations. All kinds of professional, patient, and environmental barriers may undermine clinical decision making.³⁷ Remarkably, the trial did not show an increase in the prescription of recommended medication. This lack of an effect is in contrast to the findings of a review which showed that outreach visits reduced inappropriate prescribing.⁷

Our finding that single handed practices particularly benefited from the intensive support also contrasts with a previous study by Hulscher et al⁸ which found a positive relation between partnership and improving the organisation of cardiovascular disease prevention. One possible explanation for this discrepancy is that partnerships have more people available to optimise practice organisation while GPs in single handed practices do not have to share support for clinical decision making with other GPs. Partnerships, training practices, and practices with younger GPs seem to need extra support with the provision of information and advice.

Prospective recording of patient encounters proved feasible but had some drawbacks. For some indicators, such as the prescription of angiotensin converting enzyme inhibitors and other medication to patients with heart failure, an insufficient number of decisions were available to evaluate the effectiveness of the intervention adequately. The schedule of the trial did not allow us to lengthen the recording period. In addition, the GPs generally reported 18% fewer encounters at post-intervention than at baseline, possibly due to a lack of motivation. Finally, prospective recording appears to be less suitable for the assessment of compliance with recommendations for the diagnosis of diseases and risk factors, because practitioners will obviously fail to include encounters in which they overlook the diagnosis.

Limitations

At the post-intervention measurement the GPs in the intervention group may have selectively recorded patient encounters in comparison with the control group, because the GPs were not blind to the allocation of their practice. Selective recording is nevertheless unlikely because the groups did not differ substantially in the number of patient encounters, mean age of patients, or the proportion of men. Moreover, the intervention did not improve compliance for all indicators.

Our study has several other limitations. The practices volunteered to participate and may therefore have been more interested and motivated than other practices. Furthermore, there were 25% fewer clinical decisions at post-intervention than at baseline. Bias due to the lower response at post-intervention is nevertheless unlikely because we took detailed account of the clinical situations when we assessed the compliance rates. Finally, we did not explore the effects on patient outcome. The outcome measures were nevertheless based on well accepted recommendations.

Implications for the future

Intensive support from non-physicians improved the clinical decision making for some aspects of cardiovascular care, but overall the effects were small. Research is needed to identify the barriers to change because insight into the factors which appear to prevent change will certainly help us to improve the present implementation strategy. The facilitators asked the GPs about barriers to change but may need more insight and training to be able to recognise and tackle these barriers. The effectiveness of support from non-physicians is important in terms of the costs compared with support from physicians. Furthermore, some effects of the intervention were more pronounced for certain practice characteristics. These findings suggest differences in the intensity of support required across practices and call for further research to explore the associations between practice and successful change. In our opinion, the strategy of having a non-physician to help GPs improve their clinical decision making merits further development.