Methods

Figure 1 summarises the methodology used in our study.

DATABASE

A spreadsheet was constructed, showing numbers of tests requested by all of the general practices we serve, over a six month period, for a selection of pathology tests (13 biochemistry, six haematology, eight microbiology, and one histology). These were chosen across the pathology disciplines to include approximately 95% of all tests performed by the general practitioners.

Practice list sizes were obtained from the regional health authority and applied to the database to obtain standardised requesting figures for each 1000 patients on the practice list. These results were then displayed in graphical form (fig 2). Results were anonymised by practice and collated to form a booklet, which was customised for each practice.

The practices concerned had list sizes ranging from 1000 to 16 000 patients and contained between one (four practices) and nine partners (one practice).

A brief guideline was included on each graph based on the highest amount of evidence available for the test, taken from a literature search (example shown in fig 2). Where no published evidence based guidelines were available, published consensus guidelines were used. In the absence of these, guidelines were constructed from regional audit guidelines, local multidisciplinary group guidelines, or consultant opinion, in that order of priority. These guidelines were incorporated into the booklet and linked to each graph. Specific comments on the patterns of activity for each test were also incorporated into each graph (fig 2). The first booklets were distributed at the end of March 1998. Each booklet contained a report on its front page, which was identical for each practice and listed the major findings. A baseline period was studied during the six months from April to September 1997, and compared with a second baseline period from November 1997 to April 1998.

FOLLOW UP

Numbers for each test were analysed prospectively on a quarterly basis from April, and a first follow up booklet was distributed in July 1998 for the period covering April to June 1998. A short questionnaire (fig 3) was distributed to all of the general practitioner practices six weeks after the first booklet had been circulated to gauge users' views on the benefits of the information and changes it had stimulated (fig 1), and any additional, spontaneous comments or developments generated from the booklets were recorded. A second booklet was then distributed covering June to September 1998.

STATISTICAL ANALYSES

Each time period produces a database of 616 data points (28 tests × 22 practices). Requesting numbers for each test were skewed across the general practices and are shown for one test (serum cholesterol estimation) in fig 2. Therefore, comparisons between different time periods for the same test were made by calculating the Pearson rank correlation coefficient for each test.

Interpractice variability in requesting for each test was expressed as the ratio of the numbers of tests recorded for the highest two users (top decile) to the numbers requested by the lowest two users (bottom decile). No attempt was made to compare total numbers, either for each test or by combined totals of all tests, because of the expected seasonal variation and other confounding variables on overall test requesting.

AGE AND SEX ADJUSTMENT

Age and sex adjustment was made using direct standardisation to the combined population of the 22 practices.¹⁰

Age and sex breakdowns for the 22 practices were obtained from the regional health authority. Similar distributions were obtained for three sample tests (cholesterol, glucose, creatinine + electrolytes) over the previous year from the pathology database. These data were combined to produce standardised requesting ratios, adjusted in each practice for age and sex. The ratios were then compared with the unadjusted baseline data using the same statistical methods as described above.

Results

BASELINE DATA

Data were obtained for two baseline periods, from April to September 1997 and from October 1997 to April 1998. Considerable inter-individual variation in test requesting was seen. Most pathology tests produced a skewed, S shaped activity curve (fig 2).

The median difference between the top and bottom decile of requests was 700%. There were no identifiable trends in total requesting activity.

The least variation was seen for the “core” biochemistry and haematology tests (electrolytes, liver function tests, full blood count (median 90th to 10th centile request ratio, 260%)). In contrast, intra-individual variability of test requesting for individual pathology tests was low (coefficient of variation between two baseline periods, 11%). Rankings during the first two six month baseline periods were very similar, and those differences in ranking that were observed lay in the modal band of requesters, where test numbers were numerically close together. Pearson rank correlation coefficients for the first two six month baseline periods ranged from 0.59 to 0.96 (median, 0.90) (table 1; fig 4). Within this range, however, very high correlations were found for most tests (r > 0.80 for 20 of the 28 tests), with lower correlations for the eight tests that showed the least separation between the modal practices. The differences in rank that were observed occurred in the central part of the graph where request rates were numerically close together.

INTERVENTION

The first level of intervention was the booklets themselves, which contained evidence or consensus based guidelines combined with specific comments on activity for each test. These were distributed to the general practitioners at the end of the baseline period and three months later.

Correlation coefficients between the first three month intervention period and the mean rank for the baseline periods were quantitatively similar, although slightly lower than those obtained between the two six month baseline periods. The median correlation coefficient was 0.82 (range, 0.37–0.95) compared with 0.89 (range, 0.61–0.97) for the baseline periods. Weaker correlations were found in particular for four tests (high density lipoprotein cholesterol, rheumatoid factor, ulcer swabs, and urine microscopy). Qualitatively, these four frequency distributions all contained a wide modal band that included approximately 80% of practices, in which values were numerically close together, and which also contained many tied ranks. Therefore, small differences in test requests had a disproportionately high influence on rank for these four tests. Correlation values for the second three month intervention period (not shown) were numerically similar to the first.

QUESTIONNAIRE

The questionnaire (fig 3) distributed after the baseline booklets had been released was returned by 16 of the practices (73%). Satisfaction with the information provided was high (median plus modal score, 8 out of 10) and a number of specific interventions were described by several practices. Two practices requested a visit by a consultant to discuss individual policies and two further practices requested information on additional tests not included in the initial database.

One of the two general practitioner primary care group chairs requested that a non-anonymised data listing be released for that locality, for internal use.

AGE AND SEX ADJUSTMENT

Age and sex adjustment had minimal effect on the rank positions of the different general practitioner practices for any of the three sample tests. Rank correlation coefficients between adjusted and unadjusted data for the four data sets (cholesterol, glucose, creatinine + electrolytes, and all tests combined) ranged from 0.96 to 0.99.

In addition, as for the baseline data, the only minor rank changes that did occur were seen in the central part of the graph, where request rates were numerically close together. Rankings in the upper and lower quartiles were superimposable. Sample comparative rankings for glucose are shown in fig 5.

Discussion

Most laboratories are aware that requesting patterns vary considerably between general practices, and most consultants will have witnessed the unsustained response to issuing intermittent guidance on test use, unless this is supported by structural changes—for example, changes to the laboratory request form. We have designed a benchmarking scheme for general practitioners that allows practices to compare their test requesting against other practices. This scheme incorporates local best practice guidelines in an attempt to influence requesting practice, and we describe the impact of an initial intervention based on the benchmarking document itself.

In the absence of good quality evidence across a wide range of pathology areas, benchmarking offers a potential means of assessing individual user activity, provided that the data are reproducible, that they reflect differences in clinical practice and not simply in patient demographics within the cohort studied, and that confounding variables can be excluded. Once a model has been defined, it is then possible to examine different interventions to see whether these influence practice.

The model described is straightforward to apply and repeatable, and was not significantly affected by patient demographics within the cohort studied. This model makes no attempt to correct for confounding variables relating—for example, to secondary referral patterns, which could significantly affect test requesting numbers. It might be possible for this to be done at a later date, particularly when use of the single NHS number becomes more widespread and valid patient linkage data are more accessible.

However, in keeping with other benchmarking exercises, it does highlight anomalies that the users themselves can then interpret with a view to their own practice or specialist clinics or interests, and acts as a carrier for best practice advice. The extent of the interpractice variation seen strongly suggests that differences in clinical practice play an important part in this variation, and it is difficult to explain such large differences without concluding that some users are requesting tests more appropriately than others.

The extremely close correlations between practice rankings in the two six month periods for most of the tests demonstrate that ranking practice activity provides a stable indicator that could be used to detect the impact of specific interventions aimed at influencing requesting practice. Changes in rank correlation, measured by difference plotting, should enable any important changes in practice to be identified. The poorer correlation for tests with numerically similar requesting frequencies between practices is not surprising, and suggests that the model might not be appropriate for these tests.

The initial intervention, which consisted of an information booklet containing activity and requesting guidelines, appears to have had no impact on requesting patterns. Although this finding might not be surprising to some pathologists, it does provide objective evidence that pathologists require more innovative solutions to influence requesting practice, and that the distribution of guidelines to general practitioners on an intermittent basis does not produce any medium term change in requesting habits. In a systematic review, Solomon et al confirmed that, to change physicians' test requesting patterns, interventions based on multiple behavioural factors were more successful.⁶ We are currently pursuing other proactive means of “getting the message over”, and intend to test the impact of these interventions using the same method. Interestingly, the lack of real impact on requesting activity contrasted with the positive questionnaire feedback, suggesting that the questionnaire did little to increase awareness about the data.

The fact that adjustment of requesting patterns for age and sex demographic data within the general practices has no impact on the ranks of requesters should enable us to conduct future comparisons using the raw, unadjusted data, which will simplify future analyses. Nevertheless, it cannot be assumed that age and sex will not affect data in other catchment areas. A detailed analysis of the influence of other sociodemographic factors on test activity is in preparation.

The advent of primary care groups and clinical governance leads within these groups offers the prospect that issues such as the use of pathology services might become a focus for debate within these groups. It is interesting that one of the two main primary care groups served has chosen to de-anonymise the data and share the information between its members. The success of any programme to modify use of a service will ultimately be governed by the extent to which the project is taken on internally by the users themselves, because changes to laboratory protocols and request forms cannot influence all areas of test appropriateness. Therefore, we plan to implement a range of interventions in conjunction with the primary care group clinical governance leads in an attempt to modify general practitioner requesting patterns. A second primary care group that has also received the booklet will act as a control group for these interventions. This will include discussion visits with primary care groups and a planned “cascading down” of specific questions raised at primary care group meetings to the individual practices, so that practices are encouraged by peer review to examine areas in which requesting activity is far removed from that of their colleagues. This process does not assume that a statistical outlier in terms of requesting activity is necessarily not practising correctly, but simply serves as a flag to identify possible areas to examine, specific to that practice.

These approaches are being combined with a change in the laboratory form towards requests for tests based on a diagnosis, rather than the test itself, and we wait with interest to find out whether this type of combined intervention will influence requesting of laboratory tests.