Article Text

Download PDFPDF

Better use of primary care laboratory services following interventions to ‘market’ clinical guidelines in New Zealand: a controlled before-and-after study
  1. Andrew Tomlin1,
  2. Susan Dovey2,
  3. Robin Gauld3,
  4. Murray Tilyard1,2
  1. 1Best Practice Advocacy Centre, Dunedin, New Zealand
  2. 2Department of General Practice and Rural Health, Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
  3. 3Department of Preventive and Social Medicine, Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
  1. Correspondence to Dr Susan Dovey, University of Otago, PO Box 913, Dunedin 9016, New Zealand; susan.dovey{at}otago.ac.nz

Abstract

Context Laboratory tests for inflammatory response, thyroid function and infectious diarrhoea were not being ordered as recommended by clinical guidelines.

Objective To measure changes in community laboratory-test ordering following marketing programmes promoting guidelines recommendations.

Design Controlled before-and-after study involving 2 years of national laboratory payment data before and after each intervention. Comparisons were with doctors ordering the same tests but not receiving interventions.

Setting New Zealand primary care.

Participants 3161, 3140 and 3335 general practitioners and 2424, 2443 and 2766 Comparison doctors ordering inflammatory response, thyroid function and acute diarrhoea tests from community laboratories, July 2003 to March 2009.

Interventions Three separate marketing programmes to general practitioners, each comprising written material advising of guidelines recommendations, individual laboratory-test use feedback and professional development opportunities.

Main outcome measures Number of tests, tests/doctor, patients having tests and tested patients/doctor/year before and after each intervention. Change in expenditure from before each intervention to after.

Results For Intervention doctors, erythrocyte sedimentation rate tests decreased 60.0% after the intervention; tests for C-reactive protein increased 63.1%; simultaneous erythrocyte sedimentation rate and C-reactive protein orders decreased 32.6%. Tests for free thyroxine and free triiodothyronine decreased 44.1% and 36.0%. The proportion of thyroid function tests where thyroid-stimulating hormone was the sole test ordered increased from 43.2% before the intervention to 65.2% afterwards (p<0.001; 95% CI 21.7% to 22.2%). Testing for faecal culture decreased 31.5%, giardia and cryptosporidium 31.5%, and ova and parasites 56.9%. Faecal culture as the sole initial test increased from 31.4% to 39.1% (p<0.001; 95% CI 7.2% to 8.2%). Testing by Comparison doctors changed in the same direction but with significantly less magnitude. The estimated reduction in expenditure for study tests was 23.5%.

Conclusions Clear information marketed to general practitioners improved the quality of laboratory test ordering for patients in New Zealand.

View Full Text

Statistics from Altmetric.com

Introduction

In many countries, agencies such as the National Institute for Health and Clinical Excellence develop ‘gold standard’ recommendations on preventive and clinical interventions delivered in primary care settings. Clinical guidelines for ordering laboratory tests are intended to improve patient care. Inevitably, this improved care will also foster the efficient use of healthcare resources, especially if inappropriate tests are forgone in favour of appropriate ones.

There is considerable variability in many aspects of clinical practice, including in the use of laboratory tests.1 2 Developing guidelines does not alone change practice. There is evidence that multifaceted interventional programmes are needed before guidelines recommendations have a chance of being implemented.1 3–7 Some success in changing test ordering recommended by clinical guidelines has been achieved by methods such as providing feedback to doctors on how their test utilisation and patient outcomes compare with other doctors, modifying requisition forms and changes to laboratory test funding.6 8

In this paper, we report an investigation of the effect on laboratory test ordering of three nationwide programmes in New Zealand aiming to influence the behaviour of general practitioners when ordering laboratory investigations. Table 1 shows the outcome goals and resources used in each of the three programmes.

Table 1

Laboratory-testing interventions

In July 2005, the first programme provided evidence-based guidelines for appropriate laboratory testing to assess and measure the inflammatory response. At the time, general practitioners ordered approximately double the number of tests for erythrocyte sedimentation rate (ESR) as for C-reactive protein (CRP). In most situations, CRP provides more valuable clinical information and should be the test of choice. It shows a rapid response to infection and inflammation, there are distinct ranges for normal and abnormal levels without variation for age and sex, and it is not affected by pregnancy.9 There is seldom value in requesting CRP and ESR tests simultaneously.9

A second programme about thyroid function tests began in October 2005. These tests are commonly requested in general practice, and at the time of the programme doctors requested simultaneous tests for thyroxine-stimulating hormone (TSH), free thyroxine (FT4) and free tri-iodothyronine (FT3) nearly three times more often than for thyroid-stimulating hormone alone. Screening asymptomatic patients for thyroid function is not recommended, as there is a low likelihood of developing thyroid disease when there are no obvious risk factors.10 In addition, since TSH is the most sensitive of the three tests as an indicator of thyroid function, the programme aimed to promote rational test use for diagnosing and monitoring thyroid function and to encourage the use of TSH as the sole initial test of thyroid function on most occasions.

The third programme, conducted between January and March 2008, focused on laboratory testing for enteric pathogens in the investigation of infectious diarrhoea. Although cases of vomiting and diarrhoea are quite common, most illnesses are short-lived with 50% settling by day 2 and 80% by day 4.11 The programme's key recommendations were that laboratory investigations are not routinely required for most patients with acute diarrhoea and that, if testing is indicated, a single stool specimen for fecal culture is usually appropriate. Tests for Giardia and Cryptosporidium should be requested only if there are risk factors. Testing for ova and parasites is rarely indicated.

Each programme intervention included a range of tools to increase physicians' knowledge of the guidelines and to reinforce the messages provided by the guideline recommendations. These included clinical guidelines booklets and reminder brochures, individualised laboratory utilisation reports for each general practitioner, case studies with recognition for professional accreditation processes and clinical audit packages. In each case, programme material was prepared by a panel of doctors and marketing experts, and posted to all general practitioners on the New Zealand Medical Council's register with a valid postal address (3372 general practitioners at the time of the testing for inflammatory response programme, 3358 for thyroid function testing and 3509 testing for enteric pathogens).

We aimed in this study to determine the extent to which each programme's objectives were met and estimate laboratory test expenditure before and after each intervention, and overall for the three programmes.

Methods

The programmes were developed and run by the Best Practice Advocacy Centre (BPACnz), a public university and private primary care provider collaboration established to promote ‘best’ clinical practice in primary care.

To determine whether outcome targets had been met after each programme, we analysed longitudinal data on laboratory test utilisation in the New Zealand Laboratory Claims Collection. This database, managed by the New Zealand Ministry of Health's Information Directorate, contains claims and payment information that allows central and district government agencies to monitor payments to community laboratories. For inflammatory markers and thyroid function tests, we examined all related payment data for 2 years preceding and 2 years following the 3-month intervention period. For the infectious diarrhoea programme, we examined test ordering in the 2 years preceding the intervention, but data were available for only 1 year postintervention. Ethics approval was not required.

For each programme, we calculated the mean number of tests, tests per doctor, patients having tests and tested patients per doctor per year before and after each intervention. Since some general practitioners targeted by each intervention may have retired or emigrated during the study period, only doctors who had ordered any laboratory tests in all study years preceding and after each intervention were included in the analysis. Simultaneous tests were defined as tests ordered for an individual patient on the same day by the same doctor. A series of tests for faecal culture was defined as a group of tests for the same patient and physician occurring within 14 days of the first test in the group.

All tests conducted by community laboratories and in the study database were ordered by registered medical doctors. The Intervention group comprised only general practitioners. The Comparison group included locum general practitioners not targeted by the programmes (for reasons such as not having a valid postal address on the Medical Council register) and other medical specialists whose laboratory tests are mostly carried out in hospitals rather than community laboratories. We examined their pattern of test ordering across the study period to control for external influences on ordering behaviour. Intervention general practitioners ordered 73.3% of all community laboratory tests over the combined time period covered by the analyses. Over the same period, the Comparison group of doctors ordered 20.1% of all laboratory tests.

To determine the significance of differences in the mean number of tests per doctor and patients tested per doctor before and after each intervention, we used the t test for paired measurements. The significance level was set at p=0.05 in two-sided tests. χ2 tests were used to examine differences in proportions of simultaneous tests and tests recommended as sole initial tests before and after each intervention. We estimated the total expenditure on each laboratory test before and after each intervention using the mean of the recorded amount claimed for each patient and calculated changes in net expenditure due to each programme. Appendix 1 describes how this study meets the STROBE checklist of items to be included in reports of case-control studies.

Results

Of the 3372 general practitioners who were posted programme material on testing for the inflammatory response, 3161 (93.7%) had ordered laboratory tests in all 4 years before and after the intervention and were included in the outcome analysis. Tests for ESR and CRP accounted for 5.2% of all laboratory investigations ordered by these doctors in the 2 years before the programme and 4.5% in the 2 years following. Figure 1 and table 2 show there was a 60.0% decrease in the number of ESR tests from before the intervention to afterwards. Tests for CRP increased 63.1% in the same period. Tests per doctor per year increased from 87.8 to 143.1 for CRP and decreased from 189.5 to 75.8 for ESR. The ratio of ESR to CRP testing was reduced from 2.2:1 to 0.5:1, and simultaneous testing of ESR and CRP decreased by 32.6%. Tests for CRP ordered by the 2424 Comparison doctors increased 21.4%, and their ESR tests decreased 17.8%. Changes in test use were greater in the Intervention than in the Comparison group, across all study measures. After accounting for the differences between Intervention and Comparison physicians before and after the intervention, the net effect of the intervention was a 42% increase in CRP use and a 42% reduction in use of ESR.

Figure 1

Laboratory test utilisation for the assessment of inflammatory markers. The intervention period has been shaded. CRP, C-reactive protein; ESR, erythrocyte sedimentation rate.

Table 2

Laboratory test utilisation rates before and after intervention programmes

Analysis of outcomes for the thyroid function testing programme involved 3140 (93.5%) of the 3358 general practitioners who were sent intervention materials. Thyroid function tests represented 7.6% of all tests for these doctors before the intervention and 6.4% afterwards. Figure 2 shows that although the pattern of testing for TSH did not change, tests for FT4 and FT3 decreased 44.1% and 36.0% respectively. There were significant reductions in the number of tests per general practitioner for the free hormone tests (p<0.01 in each case). Simultaneous testing of TSH and FT4 and/or FT3 reduced 41.1%. The proportion of thyroid function tests in which TSH was the sole test ordered increased from 43.2% before the intervention to 65.2% afterwards (p<0.001; 95% CI 21.7% to 22.2%). There was no significant change in the rate of TSH testing by non-intervention physicians, but tests for FT4 decreased 13.1% and FT3 14.6%.

Figure 2

Laboratory test utilisation for thyroid function. The intervention period has been shaded. FT3, free thyrosine 3; FT4, free thyrosine 4; TSH, thyroxine-stimulating hormone.

Outcome targets were evaluated for 3335 general practitioners (95.0%) sent programme guidelines on testing for enteric pathogens in the investigation of infectious diarrhoea. There were significant decreases in the number of all three targeted tests (figure 3). Testing for faecal culture decreased 31.5%, Giardia and Cryptosporidium 31.5% and ova and parasites 56.9%. These tests totalled 1.8% of all tests for these doctors before and 0.6% following the intervention. Testing for faecal culture as the sole initial test for enteric pathogens increased from 31.4% to 39.1% (p<0.001; 95% CI 7.2% to 8.2%). The proportion of culture tests requested as part of a series fell from 42.5% before the intervention to 27.0% afterwards (p<0.001; 95% CI 15.0% to 16.0%). Tests for faecal culture ordered by non-intervention doctors decreased 25.2%, Giardia and Cryptosporidium 21.1% and ova and parasites 42.8%.

Figure 3

Laboratory test utilisation in the investigation of infectious diarrhoea. The intervention period has been shaded. G&C, Giardia and Cryptosporidium; O&P, ova and parasites.

Overall, in the 2 years following the programme addressing laboratory investigations for inflammatory markers, there was a 21.1% reduction in the total number of tests for CRP and ESR, and a 25.1% saving in net expenditure, compared with the 2 years before the intervention. Net expenditure for these tests requested by Comparison doctors over the same time period decreased 5.5%, although the total number of tests for these providers represented only 19.1% of all CRP and ESR tests.

Among Intervention group general practitioners, there was a 21.2% decrease in number of thyroid function tests following the intervention and a 19.8% reduction in expenditure. Net expenditure by Comparison doctors also decreased 9.5% during this period. Tests for enteric pathogens decreased by 37.1% following the testing for infectious diarrhoea programme, and there were net savings of 36.3% of cost prior to the intervention.

Discussion

This study shows the behaviour change in general practitioners prompted by marketing campaigns to promote laboratory test utilisation in line with clinical guidelines. In the context of the total number of laboratory tests remaining largely unchanged, there were significant increases in CRP tests and reductions in the number of ESR, FT4, FT3 tests, faecal tests for Giardia and Cryptosporidium, ova and parasites, and faecal cultures. These changes were associated with a commensurate reduction in the number of patients being tested (except for the number of patients having CRP tests, which increased, and TSH, which remained the same). By contrast, during the time period covered by these campaigns, the total number of laboratory tests ordered by all physicians increased by 1% per annum, on average. The study reports the laboratory test practices of an entire nation of doctors who use community laboratory services. The programme material disseminated to Intervention general practitioners (clinical guidelines with specific, short and concise recommendations for protocols to follow when investigating inflammatory markers, thyroid function and infectious diarrhoea) included items that are likely to be readily reproducible by any national health system.

We analysed laboratory test claims data from a national data set covering the 2 years before each intervention and 2 years after, where possible, to evaluate whether utilisation rates were consistent before each programme and whether changes in referral patterns persisted after guideline material had been received. Time-series data of laboratory utilisation for CRP, ESR and thyroid function tests indicated that the effects of these interventions, each delivered over a discrete 3-month period, did not reverse in the 2 years following the programmes. The study also shows that the ratio of CRP to ESR tests ordered by Intervention general practitioners increased from 1.5:1 in the first year following the intervention to 2.4:1 in the second year following, indicating a possible flow-on effect, or building of momentum, as earlier adopters of guideline recommendations established standards of care for later adopters. Similarly, ratios of TSH to FT4 tests increased from 2.4:1 to 3.0:1 and TSH:FT3 from 7.7:1 to 8.5:1 in the 2 years following the intervention targeting thyroid function testing.

The total number of laboratory tests not targeted by the three intervention programmes changed very little during the period covered by the three campaigns. The inflammatory response and thyroid function testing programmes were essentially concurrent, and during the analysis period of these two programmes the number of tests by intervention practitioners for ESR, CRP, TSH, FT4 and FT3 combined decreased by 21.2%, while the number of all other tests ordered by the same group increased by 0.3%, suggesting that the change in targeted test use was likely attributable to the study intervention.

The study found that the clinical recommendations targeted at family physicians were adopted to a lesser degree by the Comparison group doctors. In the Comparison group, there was also a decrease in the ratio of ESR to CRP tests from 1.9:1 before the intervention to 1.3:1 following it. Simultaneous testing of TSH and FT4 or FT3 decreased 9.2% following the thyroid function testing intervention, and testing for faecal culture as the sole initial test for enteric pathogens increased from 25.1% to 31.2%. Although these were modest changes compared with those achieved by targeted general practitioners, their appearance at all might be explained by increasing general recognition of guideline recommendations or contamination of the Comparison group by doctors becoming aware of testing guidelines through peer-group discussion or reading of printed programme material located at general practices. After accounting for these and other unknown ambient effects, there was still a very substantial change in test ordering by the Intervention group that is likely attributable to the marketing intervention (a net effect of 1–46% difference in use of different tests after interventions).

The study's observational design with an opportunistic Comparison group is not ideal but is a scientifically robust evaluation approach for broad-based community intervention programmes such as the interventions in this study. The inclusion of a time-series element, as in this study, additionally strengthens the design.12 In this study, the Comparison group was different from the Intervention group not only in their non-receipt of the interventions, but also in their relatively lower use of community laboratory tests. They ordered on average 1668 tests per year from community laboratories compared with 5184 tests annually for Intervention group general practitioners. Doctors in both study groups met the study eligibility criterion of ordering tests from community laboratories in every study year. The Comparison group also included medical specialists, whereas the Intervention group comprised only general practitioners.

To our knowledge, this is the first evaluation of a programme aiming to alter trends in primary care laboratory utilisation at a national level, and of a programme deliberately adopting a marketing paradigm (rather than an educational model) to induce behaviour change. In the face of explicit calls for increasing healthcare quality, many countries are today grappling with the need for comparative effectiveness research, and particularly how this might be implemented. Intervention programmes such as those we report may offer a practical route to improving quality while containing costs.

Appendix 1

STROBE Statement: checklist of items that should be included in reports of case–control studies

Item noCompliance with recommendation
Title and abstract1(a) ‘controlled before-and-after’ in the title
(b) Provide in the abstract an informative and balanced summary of what was done and what was found. Done.
Introduction
 Background/rationale2Explain the scientific background and rationale for the investigation being reported. Done.
 Objectives3State specific objectives, including any prespecified hypotheses. Done page 6.
Methods
 Study design4Present key elements of study design early in the paper. Done page 6.
 Setting5Describe the setting, locations and relevant dates, including periods of recruitment, exposure, follow-up and data collection. Done pages 4–7.
 Participants6(a) Give the eligibility criteria, and the sources and methods of case ascertainment and control selection. Give the rationale for the choice of cases and controls. Done page 7.
(b) For matched studies, give matching criteria and the number of controls per case. Not applicable.
 Variables7Clearly define all outcomes, exposures, predictors, potential confounders and effect modifiers. Give diagnostic criteria, if applicable. Done page 6 and table 1.
 Data sources/measurement8*For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe comparability of assessment methods if there is more than one group. Done page 6.
 Bias9Describe any efforts to address potential sources of bias Done pages 11–12.
 Study size10Explain how the study size was arrived at not applicable. Comprehensive national dataset analysed.
 Quantitative variables11Explain how quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen and why. Done pages 6–7.
 Statistical methods12(a) Describe all statistical methods, including those used to control for confounding. Done pages 6–7.
(b) Describe any methods used to examine subgroups and interactions. Done pages 6–7.
(c) Explain how missing data were addressed. No missing data.
(d) If applicable, explain how matching of cases and controls was addressed. No matching.
(e) Describe any sensitivity analyses. No sensitivity analysis.
Results
 Participants13*(a) Report numbers of individuals at each stage of study—eg numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up and analysed. Done pages 5, 7–8.
(b) Give reasons for non-participation at each stage. Done pages 7–8.
(c) Consider use of a flow diagram
 Descriptive data14*(a) Give characteristics of study participants (eg, demographic, clinical, social) and information on exposures and potential confounders. Not applicable.
(b) Indicate number of participants with missing data for each variable of interest. No missing data.
 Outcome data15*Report numbers in each exposure category, or summary measures of exposure. Done table 2.
 Main results16(a) Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (eg, 95% CI). Make clear which confounders were adjusted for and why they were included. Done table 2.
(b) Report category boundaries when continuous variables were categorised. Not applicable.
(c) If relevant, consider translating estimates of RR into absolute risk for a meaningful time period. Not applicable.
 Other analyses17Report other analyses done—eg, analyses of subgroups and interactions, and sensitivity analyses. Not applicable.
Discussion
 Key results18Summarise key results with reference to study objectives. Done pages 10–11.
 Limitations19Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both direction and magnitude of any potential bias. Done pages 11–12.
 Interpretation20Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies and other relevant evidence. Done pages 10–12.
 Generalisability21Discuss the generalisability (external validity) of the study results. Done pages 11–12.
Other information
 Funding22Give the source of funding and the role of the funders for the present study and, if applicable, for the original study on which the present article is based. Done page 12.
  • An Explanation and Elaboration article discusses each checklist item and gives methodological background and published examples of transparent reporting. The STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) checklist is best used in conjunction with this article (freely available on the websites of Public Library of Science Medicine at http://www.plosmedicine.org/, Annals of Internal Medicine at http://www.annals.org/ and Epidemiology at http://www.epidem.com/). Information on the STROBE Initiative is available at http://www.strobe-statement.org.

  • * Give information separately for cases and controls.

  • References

    View Abstract

    Footnotes

    • AT and MT had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. The study was completed as part of the ongoing evaluation by BPACnz of its programmes. BPACnz has five shareholders: IPAC, Pegasus Health, ProCare Health, South Link Health, and the University of Otago. BPACnz is contracted to the Pharmaceutical Management Agency (PHARMAC), the government agency responsible for national pharmaceutical subsidy expenditure and District Health Boards New Zealand (DHBNZ), to market evidence-based recommendations and guidelines to New Zealand's general practitioners on prescribing and laboratory test investigations, respectively. Study data were accessed as part of the DHBNZ contract, but the research was conducted independently of DHBNZ.

    • Funding This research was completed as part of the authors' employment in Best Practice Advocacy Centre (BPACnz) or its stakeholder organisations.

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.