Concordance with urgent referral guidelines in patients presenting with any of six ‘alarm’ features of possible cancer: a retrospective cohort study using linked primary care records

Background Clinical guidelines advise GPs in England which patients warrant an urgent referral for suspected cancer. This study assessed how often GPs follow the guidelines, whether certain patients are less likely to be referred, and how many patients were diagnosed with cancer within 1 year of non-referral. Methods We used linked primary care (Clinical Practice Research Datalink), secondary care (Hospital Episode Statistics) and cancer registration data. Patients presenting with haematuria, breast lump, dysphagia, iron-deficiency anaemia, post-menopausal or rectal bleeding for the first time during 2014–2015 were included (for ages where guidelines recommend urgent referral). Logistic regression was used to investigate whether receiving a referral was associated with feature type and patient characteristics. Cancer incidence (based on recorded diagnoses in cancer registry data within 1 year of presentation) was compared between those receiving and those not receiving referrals. Results 48 715 patients were included, of which 40% (n=19 670) received an urgent referral within 14 days of presentation, varying by feature from 17% (dysphagia) to 68% (breast lump). Young patients (18–24 vs 55–64 years; adjusted OR 0.20, 95% CI 0.10 to 0.42, p<0.001) and those with comorbidities (4 vs 0 comorbidities; adjusted OR 0.87, 95% CI 0.80 to 0.94, p<0.001) were less likely to receive a referral. Associations between patient characteristics and referrals differed across features: among patients presenting with anaemia, breast lump or haematuria, those with multi-morbidity, and additionally for breast lump, more deprived patients were less likely to receive a referral. Of 29 045 patients not receiving a referral, 3.6% (1047) were diagnosed with cancer within 1 year, ranging from 2.8% for rectal bleeding to 9.5% for anaemia. Conclusions Guideline recommendations for action are not followed for the majority of patients presenting with common possible cancer features. A significant number of these patients developed cancer within 1 year of their consultation, indicating scope for improvement in the diagnostic process.


INTRODUCTION
In 2015, the US Institute of Medicine report 'Improving Diagnosis in Health Care' highlighted the need to improve the quality and safety of diagnosis. 1 It noted that 'there are few clinical practice guidelines available for diagnosis', contrasting this situation with the large number of guidelines aimed at improving the management of patients with a diagnosed illness. Guidelines for the management of patients with symptoms of possible cancer, which have been introduced in some health systems such as the English National Health Service (NHS), represent an important exception warranting investigation. [2][3][4] The period between the patient's first symptomatic presentation and cancer diagnosis is important, as shorter diagnostic intervals may contribute to an earlier stage at diagnosis and better patient experience. [5][6][7][8] In order to improve the quality of diagnostic care and reduce delay, guidelines introduced in England in 2000 recommend that patients who present with certain features of possible cancer should be referred to hospital services by their GP for specialist assessment within 2 weeks. Relevant 'redflag' features for which urgent referrals are recommended were defined by the National Institute for Health and Care Excellence (NICE) initially in 2005 3 4 and updated in 2015 mainly in order to lower the cancer risk threshold for inclusion of features to 3%. 9 The guidelines Original research include presenting features, which cover clinical signs such as breast lump, test results such as iron-deficiency anaemia, and symptoms such as haematuria.
Guidelines can only ever be as effective as the degree to which they are implemented. The implementation of the Two Week Wait guidelines has been associated with a continuous increase in the number of patients (with or without cancer) who are investigated expeditiously, and in improvements in care outcomes in patients with cancer. [10][11][12][13][14] While this evidence indicates that the Two Week Wait guidelines are helping to improve diagnostic processes and outcomes in England, the extent to which they are adhered to is unknown. Other evidence suggests that there is some variation in the quality of diagnostic care in cancer patients initially presenting to primary care, [15][16][17][18] the use of endoscopic investigations between GP practices, and the proportion of cancer patients who were diagnosed after a Two Week Wait referral. 16 19-21 Furthermore, previous observed inequalities in stage at diagnosis, referral interval and survival [22][23][24][25][26][27] may suggest an association between certain patient characteristics (such as age, gender, socioeconomic status and ethnicity) and the risk of guideline non-adherence.
In principle, guideline-discordant referral behaviour may reflect accurate clinical judgement resulting in no adverse clinical outcomes for patients. How often this may be the case is, however, unknown, as studies thus far typically examined selected patients, such as from case series of malpractice claims including patients known to have been harmed. [28][29][30] There is little evidence about the outcomes of non-adherence in population-based data. In this paper we focus on concordance with referral guidelines for patients presenting with features of possible cancer. Our research questions are:

Patient and public involvement
A patient and public involvement group comprising five members met two times throughout the study period. The group was asked to reflect on the study design and research questions. Results, their meaning, and potential explanations were discussed at several stages. They contributed to identifying explanations for GP referral decision making and differences in referral level between patient groups.

Data analysis
The index consultation was defined for each patient as the first consultation with one of the six features. For these patients, referrals were identified in the HES dataset. Referral in HES is linked to the date the referral was received by the hospital, and recorded as 'Routine', 'Urgent', 'Two Week Wait' or 'Unknown'. As the NICE guidelines define an urgent referral as a referral to see a specialist within 2 weeks, an outcome of urgent referral was defined as a referral having been made in the 14 days after the index consultation with a recorded urgency of 'Urgent' or 'Two Week Wait'. The choice for this definition was pragmatic, acknowledging that there may be administrative delays between a consultation occurring and a referral being made, and was supported by our data showing that most patients referred for an urgent assessment were referred within 2 weeks of the index consultation (online supplemental appendix A, figure  A1). Existing feature code lists were used to identify presentations with features of interest. These code lists were developed using robust methods 46

and have
Original research been used successfully in previous studies. 38 47 Patient age, gender, comorbidities, and previous cancer diagnosis were extracted from CPRD. Age was treated as a categorical variable with an 18-to 24-year-old group, 10 year age groups between 25 and 84, and an 85 and older group. Comorbidities were conditions included in the Quality of Outcome Framework 2015/2016, using existing code lists applied to all CPRD data before a patient's index consultation. A simple count of comorbidities was used, grouping those with four or more comorbidities together. National quintiles were used to define IMD groups. After describing the proportion of patients receiving an urgent referral, we investigated which patient groups were more or less likely to receive an urgent referral using multilevel logistic regression. An initial main effects-only model included patient age, gender, deprivation level, number of comorbidities, feature type, previous history of cancer, and a random intercept for referring clinician and general practice to account for clustering of patients within referring clinicians within practices. Between clinician and practice variation is quantified using the odds ratio covering the 95% mid-range of practices calculated from the estimated random effect variances. 48 Reference categories were based on the highest number of patients with all genders and features represented. Interaction terms between patient characteristics and feature type were investigated individually with all significant interactions retained in the final model. We excluded 18 cases due to missing data on deprivation level. Significance of categorical variables was tested with a joint Wald test. The study power calculation assumed at least 8000 patients presenting with each feature, which would provide 90% power (p=0.05) to see a 4% absolute change from 25% discordant care in groups comprising 20% of the sample for any one feature (for example, 29% discordant care in the most deprived fifth of patients compared with 25% in others).
Finally, we investigated how many patients were diagnosed with cancer within 1 year of their index consultation by whether they received an urgent referral or not. Cancer incidence was based on the presence of a tumour in the NCRAS data with an International Classification of Diseases, 10th revision (ICD-10) invasive neoplasm code (C00 to C97 but excluding non-melanoma skin cancer, C44) and a date of diagnosis within 1 year of the index consultation.

Sensitivity analyses
We performed five sensitivity analyses. First, we repeated the analysis excluding urgent referrals and solely focusing on Two Week Wait ones, to assess whether including urgent referrals had an impact on the findings. Second, we examined the impact of our restriction that referrals must occur within 2 weeks of the index consultation by expanding this window to 90 days. Third, the main analysis was repeated after also adjusting for ethnicity (extracted from HES; coded as white, black, Asian, mixed, and other ethnicity), which was not included in our main analysis due to a considerable amount of missing data (38%). Fourth, in addition to referrals captured in HES we also considered referrals recorded in CPRD. Where a record of a referral flagged as 'Two Week Wait', 'Red flag' or 'Urgent' in either CPRD or HES was made within 2 weeks of first presentation, the patient was considered to have had an urgent referral. This sensitivity analysis allows for the possibility that GPs made referrals which were not recorded in HES (either because changes were made to the referral after it was made or because the referral was not captured by HES, for example, for an investigation outside of the outpatient setting). Fifth, a Continuous registration at the same general practice from a year before to a year after their first index consultation. ‡ This table was created by the authors. *There were minor changes in the recommendations for patients presenting with irondeficiency anaemia and haematuria between the original (2005) and updated (2015) guidance. Therefore, we have restricted analysis to patients where an urgent referral would have been recommended in both sets of guidelines. Specifically for anaemia, inclusion as a first presentation was based on the more stringent test values included in the 2005 NICE guidelines of a haemoglobin level ≤11 g/dL for men and ≤10 g/ dL for women, plus a ferritin level <20 ng/mL and/or a mean red cell volume <80 fL, and age restriction of 60 years and over related to the 2015 NICE guidelines. For haematuria, the 2015 NICE guidelines advise referral only if there is no evidence of a urinary tract infection, or if haematuria recurs or persists after successful treatment of a urinary tract infection. As such, the first and/or second consultation where antibiotics were prescribed in the absence of referral were excluded. Third visits within 6 months of the first visit were included regardless of GP treatment and referral decision-making.
†Although the guidelines do not specify an age range for post-menopausal bleeding, we exclude patients under the age of 45 years due to the small number in our sample (n=30). ‡With the exception of patients who presented with haematuria, the index consultation was defined for each patient as the first consultation with one or more of the six features based on medical codes for five features, and test results for irondeficiency anaemia. For some patients presenting with haematuria, the second or third visit was included as their index consultation instead of the first visit, but exclusion criteria were applied to their first visit. CPRD, Clinical Practice Research Datalink; GP, general practitioner; NICE, National Institute for Health and Care Excellence.
Original research sensitivity analysis was performed including neoplasm in situ (ICD-10 codes D00 to D48) in addition to invasive neoplasms. All analyses were conducted using the statistical software Stata v14.2. 49

Patient characteristics
There were 48 715 index consultations by patients with a feature of interest where a Two Week Wait referral would have been recommended (table 2). Among these patients, the most common presenting features were breast lump (33%) and rectal bleeding (27%). The mean age of the included patients was 60.4 years (range 18-104; SD 15.6). However, age ranges varied by feature, with the lowest mean age of 49.5 years (range 30-104 years; SD 13.7) for patients with breast lump and the highest mean age of 77.9 years for patients with anaemia (range 60-102; SD 9.1). Most patients had at least one comorbidity (80%). Patients who lived in an area classed as the lowest quintile (least deprived) of the IMD were over-represented (26% vs an expected 20%).

Urgent referrals
Overall, 40% (n=19 670) of patients received an urgent referral within 2 weeks of visiting the GP. The percentage of urgent referrals varied greatly by presenting feature, ranging from 17% (n=1384) for patients with dysphagia to 68% (n=11 007) for patients with breast lump (table 3).

Associations between patient characteristics and urgent referrals
The main effects-only models showed evidence that age (p<0.001), feature type (p<0.001), and comorbidities (p<0.001) were associated with the probability of receiving an urgent referral (online supplemental appendix A, figure A2;    Original research patient's presenting features. Although there was no evidence for an average effect of deprivation across all features, there was evidence that patients living in more deprived neighbourhoods were more likely to be referred urgently if they presented with haematuria, post-menopausal bleeding, or rectal bleeding, though the opposite was true for women presenting with breast lump (online supplemental appendix A, figure A3(a)). Furthermore, patients presenting with anaemia, breast lump or haematuria were less likely to receive an urgent referral if they had a higher number of comorbidities, though the gradient was stronger in patients with anaemia than for other features (online supplemental appendix A, figure A3(b)). Finally, although the association between age and urgent referral was similar for all patients, the size of age variation was greater for patients presenting with dysphagia and breast lump (online supplemental appendix A, figure  A3(c)). However, this largely reflected the larger age range covered by referral guidelines for those features. Sensitivity analyses excluding the 58% of patients who received an urgent as opposed to Two Week Wait referral (online supplemental appendix A,

Statement of principal findings
Six out of 10 patients presenting to primary care with a high-risk feature of possible cancer did not receive an urgent referral in the 14 days after presentation, despite this being a guideline-recommended action. Urgent referral frequency varied by feature, with patients with breast lump receiving the highest percentage of referrals and patients with dysphagia the lowest. Younger patients, and those with comorbidities were less likely to receive an urgent referral. Associations between patient characteristics and urgent referrals differed by feature. More deprived women with breast lump, and patients with anaemia, breast lump, or haematuria and multi-morbidity were less likely to receive a referral; 3.6% of patients who did not receive an urgent referral were diagnosed with cancer within 1 year, this percentage varying between 2.8% for rectal bleeding and 9.5% for iron-deficiency anaemia.

Strengths and limitations
We used a large, longitudinal, validated linked dataset which has been used extensively for cancer diagnostic studies, 10 50 51 enabling important insights into patients' journeys through the healthcare system. However, there are limitations. First, as an urgently referred patient would usually be investigated in an outpatient setting, the HES dataset comprised outpatient hospital data. Consequently, patients referred and admitted to hospital or directed to an emergency department will not have been identified even though timely action was taken. However, the number of such patients is likely small for the studied presenting features, especially given the substantial decrease in the number of cancers diagnosed following emergency GP referrals in the last decade. 13 Second, identification of index consultations with one of the six features was based on medical codes in patient records. Some patients may have been missed because the feature was only recorded in inaccessible 'free text' (which is not available to researchers to avoid de-anonymisation) or because the feature was not recorded at all. However, CPRD studies of free-text data suggest it usually only confirms coded entries. 52 Consequently, some patients with features of interest may not have been included in the study. However, those who were included almost

Original research
certainly had the feature, and thus a Two Week Wait referral would have been recommended. Third, inclusion of patients was limited to 2014 and 2015 as cancer registry data were only available up to and including 2016 at the time of data extraction. However, it is unlikely that this will have affected results, as the guidelines had been in use for a number of years, and analyses were restricted to patients where an urgent referral would have been recommended in both the 2005 and 2015 guidance. Fourth, we excluded patients with haematuria who were treated for possible urinary tract infection (unless it was not the last visit or a third consultation within 6 months) on the basis of prescriptions for two of the most common first-line antibiotics for urinary tract infection. A small number of patients will have been prescribed different antibiotics and been included in error. Fifth, although the study showed that many patients were diagnosed with cancer after not receiving an urgent referral after presenting with a suspected cancer feature, we were unable to give insight into the potential impact of nonreferral on cancer stage. Finally, we note that although for three features we exceeded our planned sample size, the number of patients presenting with iron-deficiency anaemia or post-menopausal bleeding was significantly less than planned.

Comparison with existing evidence and meaning of the study
Previous studies have reported that the number of GP consultations before referral varied by cancer type. 53 It appears that GPs are less likely to suspect cancer for some features compared with others. This may be partially explained by the greater likelihood of some features being caused by other explanations than cancer. However, the risk of cancer with all these features is always low in absolute terms, with iron-deficiency anaemia having the highest positive predictive value of the six for cancer (for men over 60 years with a haemoglobin <11 g/dL and features of iron deficiency, the positive predictive value is 13% 54 ). The decision to refer may be influenced by factors other than guidelines, such as GPs' symptom interpretation, 55 additional presenting features supporting a different diagnosis, and clinical intuition, 56 but also by how local health services are organised. 20 Variation in referral has been shown to be partly attributable to Clinical Commissioning Groups (CCGs) and Acute Hospital Trusts. 20 This is supported by qualitative research suggesting that GPs are hesitant to refer or even feel pressured by CCGs to not refer due to resource pressures. 57 Additionally, although probably only responsible for a small proportion of non-referrals, there are a number of other factors which may affect whether a referral was made or recorded (box 1). Regardless of the GPs' reasons for referral or non-referral, our study shows that GPs often made the right decision regarding referral for their patients. Given the proportion of patients going on to be diagnosed with cancer was considerably higher in those receiving an urgent referral than those who did not, we can conclude that GP referral decision-making is not without value. However, given the number of patients diagnosed with cancer after non-referral, we may question whether clinical judgement is good enough: 5.5% of patients with anaemia not receiving an urgent referral were diagnosed with colorectal cancer within 1 year, 3.5% of women presenting with breast lump who did not receive a referral were diagnosed with breast cancer, and 2.9% who presented with post-menopausal bleeding were diagnosed with uterine cancer. In these patients it can be argued that guideline-discordant decision-making may have resulted in a missed opportunity to diagnose early. Better adherence to the guidelines may therefore be important in order to increase detection rate, even for alarm features with already high urgent referral rates.
Our finding that younger patients were less likely to receive an urgent referral reflects earlier research reporting that younger patients typically experience longer diagnostic timelines. 53 The present study also offers new insights into how multi-morbidity may affect diagnostic timeliness. Although there are suggestions that more contact with health services can shorten diagnostic intervals, a growing body of evidence suggests that multimorbidity can also prolong diagnostic intervals. 15 58-61 Given our findings, it may be these prolonged intervals arise, in part, due to a lower likelihood of receiving an urgent referral. Additionally, research suggests that multimorbidity is associated with decreased use of specialist investigations. 62 This may explain the strong multimorbidity gradient observed for patients with anaemia where urgent referral would lead to an invasive test.

Box 1
Mechanisms affecting referral Although we expect that it only affects a small number of patients, there are a number of mechanisms besides GP referral decision-making which may potentially influence whether an urgent referral was made or recorded: ⇒ Patients were admitted to hospital via emergency admission ⇒ The referral was not accepted by the hospital. (This should be captured by the sensitivity analysis including Clinical Practice Research Datalink (CPRD) referrals) ⇒ The patient refused to be referred ⇒ A downgrade of the urgency level of the referral was requested by the hospital. (This should still be captured in the CPRD sensitivity analysis) ⇒ Index consultation took place with out-of-hours practice services ⇒ Variations in local guidelines for referral. (For most patients this should be captured in either the sensitivity analysis including CPRD referrals or the sensitivity analysis including referrals made up to 90 days after presentation) ⇒ The patient received a related referral before first presentation, which affected the decision to refer

Original research
Future research investigating how multi-morbidity affects GP referral decision-making for potential cancer features may help target improvement efforts. Although, on average, deprivation was not associated with urgent referrals, more deprived patients with haematuria, rectal or post-menopausal bleeding were more likely to receive a referral. Although we can only speculate, this finding could be explained by earlier research suggesting that deprived patients are more likely to delay presentation, 63 64 resulting in more serious potential cancer features, increasing the chance of an urgent referral. On the other hand, more deprived women presenting with breast lump were less likely to receive an urgent referral. As few women with breast cancer delay presentation, 63 the association between deprivation and referral is likely to reflect differences post-presentation. Patients with a higher socioeconomic status tend to be more effectively able to communicate their symptoms and concerns, 65 66 while GPs' communication tends to be more patient-centred with less deprived patients. 67 This may potentially result in closer alignment in perceptions of symptom significance 68 and influence GPs' decision to refer.
We have identified patient groups who may be at risk of longer diagnostic timelines. GPs may be less likely to refer patients when their age, 69 or alternative medical explanations, suggest a lower risk of cancer. However, guidelines incorporate patient age and thus recommended action is still appropriate for those age groups.
Clinical practice guidelines have been shown to improve treatment quality for a range of conditions and could also help to improve the quality of the diagnostic process. However, our study shows that recommendations for the assessment of patients with features of possible cancer are not always followed. Stricter adherence to the guidelines and increased awareness of patient groups especially at risk of long diagnostic timelines may help improve early diagnosis and ultimately cancer survival rates. Due to the potential impact of regional health services, interventions to reduce guideline discordant behaviour may have more impact if they do not just focus on GPs and individual practices, but also on local diagnostic service provision.