Article Text

Download PDFPDF

Influencing referral practice using feedback of adherence to NICE guidelines: a quality improvement report for dyspepsia
  1. Glyn Elwyn1,
  2. Diane Owen4,
  3. Llinos Roberts1,
  4. Kathie Wareham2,
  5. Paul Duane2,
  6. Miles Allison3,
  7. Alan Sykes4
  1. 1Department of General Practice, Centre for Health Sciences Research, Cardiff University, Cardiff, UK
  2. 2Morriston Hospital, Swansea NHS Trust, Swansea, UK
  3. 3Department of Adult Medicine, Royal Gwent Hospital, Gwent Hospital Trust, Newport, UK
  4. 4Primary Care Group, Grove Building, Swansea University, Swansea, UK
  1. Correspondence to:
 Professor G Elwyn
 Department of General Practice, Centre for Health Sciences Research, Cardiff University, Heath Park, Cardiff CF14 4YS, UK; elwyng{at}


Problem: Rising demand and increasing waiting times for upper gastrointestinal endoscopy (gastroscopy).

Design: Quality improvement study with pre- and post-intervention data collection.

Setting: Three endoscopy units in two hospital trusts (Singleton, Morriston and Baglan Hospitals endoscopy units), UK.

Key measures for improvement: Number of gastroscopy requests from general practitioners (GPs) and hospital doctors; their adherence to dyspepsia referral guidelines and the referral-to-procedure interval for upper gastroscopy. Data collected for six months before and for five months after the intervention.

Strategy for change: Referrals were assessed against the National Institute for Health and Clinical Excellence (NICE) guidelines for the management of dyspepsia by two part-time GPs and feedback sent to clinicians where requests did not adhere to the referrals criteria

Effects of change: Adherence to guideline criteria increased significantly among GPs after the intervention (from 55% to 75%). There was no similar effect for hospital doctors, although their adherence rate (70%) was at a higher level than that of GPs before the intervention. The number of gastroscopy referrals for dyspepsia declined after the intervention, particularly from hospital doctors where a drop of 31% was observed, from 26.6 to 18.4 referrals per week. With the inclusion of seasonal effects, an estimated drop of 3.2 referrals per week from general practice was not significant (p = 0.065) while an estimated drop of 10.0 referrals per week for hospital doctors was very significant (p<0.001).

Lessons learnt: Referral assessment can be successfully introduced and shows promise as a way of improving the quality of referrals and reducing demand. Hospital clinicians are more resistant than GPs to referral assessment but nevertheless responded to the feedback by reducing their endoscopy gastroscopy requests. Most such referrals are generated in hospitals rather than in primary care: this finding has important implications for demand management.

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

The three hospitals included in this quality improvement study were struggling to meet a rising demand for endoscopy, particularly for the investigation of upper gastrointestinal symptoms—a situation mirrored widely across the UK. This was leading to long waiting lists; in one of the endoscopy units the waiting time for a “non-urgent” endoscopy was quoted to be in the region of 12 months. Reviews, supplemented by an individual patient date meta-analysis, have shown that prompt endoscopy is not a cost-effective strategy for the initial management of dyspepsia.1,2 In addition, the delay between referral and diagnostic procedure led to concerns that patients were not receiving a good service and that unacceptable delay in diagnosing problems such as oesophageal and gastric cancer would occur. The NHS Improvement Plan (June 2004) set a new overall goal for the NHS—that by March 2007, all scans and diagnostic procedures would have been accomplished within 13 weeks of general practitioner (GP) referral.3 When interviewed, the gastroenterologists performing the majority of the procedures felt that part of the problem was the lack of adherence to recognised guidelines. A specific example was the situation when endoscopy was requested for the investigation of dyspepsia in a clinical situation where guidelines, widely debated and released to the NHS by the National Institute for Health and Clinical Excellence (NICE),4 advised other measures, such as testing and eradication for H pylori.

Three endoscopy units were included in the study: two units, based at Morriston and Singleton hospitals, are part of the Swansea NHS Trust and the third unit, based at the Baglan Bay hospital, part of the Mid Glamorgan NHS Trust. The hospitals have 850, 600 and 270 beds respectively and the endoscopy units are managed by gastroenterologists, although surgeons also undertake endoscopies and arrange lists independently.

The focus of this study was on patients referred for an gastroscopy where the presenting complaint was dyspepsia. Although the evidence is sparse,5 there have been reports that referral assessment at the primary and secondary care interface can successfully influence both the quality and the volume of demand.6,7,8,9,10 We therefore designed a quality improvement intervention that aimed to introduce referral assessment in order to change the proportion of referrals that adhered to accepted guidelines, and to assess what impact this might have on demand for endoscopy and on the referral-to-procedure interval.


We chose three metrics for the evaluation of the proposed intervention:

1. Adherence to NICE guidelines for the referral of patients with dyspepsia, by GPs and doctors working in the hospitals.

2. The number of referrals received for gastroscopy.

3. The referral-to-procedure interval (in days) for gastroscopy at the three endoscopy units.


We conducted an uncontrolled before and after study. Copies of all requests for upper gastrointestinal endoscopy (irrespective of clinical problem) were collected at the three endoscopy units for six months before the intervention start date (1/11/04) and for five months afterwards. Data from all the referrals were recorded on a database and coded according to the problems described. Two GPs were employed on a part-time basis to judge whether the requests for gastroscopy adhered to the NICE referral guidelines,4 using a standardised protocol (available from the author). The interval, between the date on the referral letter and the date of the endoscopy was calculated in days in order to assess the interval between request and procedure.


We analysed the data as follows:

1. Logistic regression modelling was used to analyse the data regarding adherence to guideline criteria, using period of referral, origin of the referral (GP and hospital doctors) as qualitative predictor variables and time as a quantitative predictor variable. While the data we report have been aggregated across the three units, we also conducted the same analysis at the level of the individual endoscopy units.

2. Poisson regression models were applied to analyse weekly rates of referrals for upper gastroenterological endoscopy by their origin (general practice or hospital clinician) where dyspepsia was the prime reason, and also all referrals, irrespective of presenting problem). By including the month of referral as a qualitative variable, seasonal effects were estimated and controlled.

3. The interval, in days, between the referral date and the date when endoscopy was performed (referral-to-procedure interval) and a two-sample z-statistic for the difference in means between the pre- and post-intervention periods calculated.

Preliminary analysis revealed that the majority of referrals for upper gastroenterological endoscopy originated in hospitals and that multiple lists for gastroscopy were in existence—that is, surgeons commonly conducted independent gastroscopy lists, making it difficult to introduce standardised pathways to handle referrals within a provider organisation.


The intervention was composed of the following elements. In October 2004, a letter was sent to all 215 GPs in the catchment area of the three endoscopy units. The same letter was also sent to the 359 hospital consultants and post holders at junior grades based at the three hospitals. The letter stated that as from 1 November 2004 two GPs would be employed part-time to assess all endoscopy letters and that referrals where the prime problem was dyspepsia would be judged against the recently issued NICE guidelines for the management of dyspepsia in primary care. The letter went on to say that where referrals did not adhere to the guidelines, the referring doctor would be informed by letter, giving the reason for the non-adherence accompanied by a synopsis of the relevant guidance. Requests for gastroscopy where dyspepsia was not the stated clinical problem were recorded but analysed separately. The correspondence made it clear however that although all referrals for endoscopy were assessed by a clinician there would be no interference to the referral pathway. Referrals were therefore processed in the usual way and no delay or diversion was instigated. An All Wales Dyspepsia Management Guideline, closely modelled on the NICE and SIGN guidelines,4,11 had been circulated to all clinicians in NHS Wales two weeks before the start of the intervention. This intervention was in effect occurring against a background of increased awareness that there were changes in the criteria for what was considered “appropriate” referral for gastroscopy, and a general awareness that endoscopy waiting times were increasing.


1. Adherence to NICE guidelines

Aggregated measures over the three units showed that GPs increased their adherence rates from a mean 55% before the intervention to 75% during the intervention period (95% confidence interval (CI) 13.6% to 26.4%, p<0.001). This was a significant period effect (p<0.001). This change for GPs was observed at all three endoscopy units (Singleton 52% to 71%, Morriston 66% to 80%, Baglan 52% to 71%). The mean adherence rate for doctors working in hospital was higher than that for GPs and stood at 70%. Although there appears to be no step-change corresponding to the period change, there appears to be a significant trend upwards in the adherence rate for hospital doctors after the intervention started. Figure 1 illustrates the rates of adherence for the time period and fitted models.

Figure 1

 Adherence of gastroscopy referrals to guidance: weekly proportions (%).

2. Numbers of referrals for gastroscopy to investigate dyspepsia

A decline in the number of referrals for gastroscopy was observed in each of the three units. Figure 2 shows weekly referral totals for GPs and hospital doctors (dyspepsia referrals and all referrals) and include 12-week moving averages. For dyspepsia referrals, a decline in average weekly referral rate from 13.3 to 12.1 referrals per week for GPs and 26.6 to 18.4 referrals per week for hospital doctors (or the equivalent figures 14.5 to 14.1 and 51.9 to 51.0 for all referrals) may be confounded by seasonal effects. To control for this, a monthly categorical predictor variable was introduced and Poisson regression models were fitted to the data. For the dyspepsia referrals, an estimated reduction of 3.2 referrals per week for GPs was not significant (p = 0.065), while for hospital doctors, the corresponding estimated decrease of 10 referrals per week was very significant (p<0.001). For all referrals, there was no change in the weekly rate across the two time periods for GPs or hospital doctors. To summarise, the greatest impact on referral volume is on the number of referrals from hospital doctors for gastroscopy to investigate dyspepsia, a drop from 26.6 to 18.4 referrals per week, a decrease of 31%.

Figure 2

 Referrals requesting gastroscopy: weekly totals.

3. The referral-to-procedure intervals

The data show that the intervention, although focused on referral for dyspepsia, had the effect of significantly reducing the referral-to-procedure interval for gastroscopy. The mean interval in the pre-intervention period (for all referrals) was 52.1 (sample size 1188, SD 67.9) days, compared to a mean interval of 39.4 (sample size 612, SD 46.2) days in the post-intervention period. A two sample z-statistic for this difference in the means is 4.14, p<0.001. The 95% CI for the mean difference is 6.6 to 18.6 days.


The results show that it is feasible to employ GPs to assess the referrals of their colleagues and to send feedback to individual clinicians where requests do not meet guidelines. Although we do not have control data, the results show a significant change in adherence rates by GPs after the intervention was introduced, which was sustained over our data collection period, and occurred in all three endoscopy units. We also observed a 31% drop in the referrals from hospital doctors for dyspepsia overall, with an estimated drop of 10 referrals per week when allowing for seasonal changes. We consider these to be important, if preliminary, findings that should be investigated further by a controlled trial.

The intervention provoked resistance from some clinicians: we received 22 letters, 21 from specialists (14 letters from 7 surgeons, and 7 letters from 5 physicians) and one from a GP. These letters were critical of the referral assessment strategy. The strongest concern was a perceived erosion of clinical freedom. This view was also commonly associated with an outright disagreement with the NICE guidelines,4 often viewing them as mechanisms to “ration” services. A few noted that the guidance used in the study was developed for primary care, and not for dyspepsia occurring in hospital settings.

We have learnt the following lessons. Firstly, although referral assessment can be introduced elements of resistance will emerge. Secondly, the method shows promise as a way to improve referral quality, reduce demand and reduce the referral-to-procedure interval. Thirdly, 1720 patient beds in three hospitals and 359 hospital doctors generated more gastroscopy referrals per month than 215 GPs serving a population of approximately 360 000 patients, an underresearched area of clinical practice.12 Any system wishing to introduce demand management in the area of endoscopy referrals will need to consider the demand generated by the total healthcare system and not just referrals from primary care.13

Key messages

  • Clinician referral assessment and feedback can be successfully introduced where it does not threaten clinical processes.

  • Providing feedback from referral assessment shows promise as a way to both improve the quality of referrals from primary care and to reduce demand in general.

  • There is a greater demand for gastroscopy in hospitals than in primary care. Demand management systems need to consider the work generated by secondary care and not just referrals from primary care.

  • The results of this before and after study justify further investigation in the form of a randomised controlled trial.


We wish to thank all the general practitioners and hospital based doctors for participating in this study and the steering group members, Andrew Bellamy, Lindsay Davies, Maxine Evans, Richard Evans, Ceri Gimblett, Darren Griffiths, Clem Lai, Adrian Pelk, Mesbah Rahman, Julie Rees, Linda Reid, Teryl Stoneman, Linzi Thomas, Marilyn Wilkinson, John Williams and Tudor Young for their work.



  • Funding: National Leadership and Innovation Agency for Healthcare, Welsh Assembly Government.

  • Competing interests: None declared.

  • Ethics approval: The Bridgend, Neath Port Talbot and Swansea Local Research Ethics Committee approved the study (reference 04/WMW02/4).

    Contributors: GE proposed and designed the study, obtained the funding, acted as principal investigator and wrote the article. KW and CD (Clinical Research Unit) managed the data collection and managed the project. DO and LR undertook the referral assessment. MA and PD prepared the all-Wales guidelines and provided specialist gastroenterological advice. AS undertook the statistical analysis. All the authors contributed to the drafting and editing of the final paper. GE is the guarantor.