You are here

  • Home
  • Archive
  • Volume 38, Issue 10
  • Variation in referral rates to emergency departments and inpatient services from a GP out of hours service and the potential impact of alternative staffing models

Article Text

Article menu
XML
Original research
Variation in referral rates to emergency departments and inpatient services from a GP out of hours service and the potential impact of alternative staffing models
  1. Daniel Lasserson1,2,
  2. Honora Smith3,
  3. Sophie Garland4,
  4. Helen Hunt4,
  5. Gail Hayward5
  1. 1Faculty of Medicine, Division of Health Sciences, University of Warwick, Coventry, UK
  2. 2Department of Acute Medicine, Sandwell and West Birmingham Hospitals NHS Trust, Birmingham, UK
  3. 3Faculty of Engineering Science and Mathematics, Department of Mathematical Sciences, University of Southampton, Southampton, UK
  4. 4Oxford Health NHS Foundation Trust, Oxford, UK
  5. 5Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK
  1. Correspondence to Professor Daniel Lasserson, Division of Health Sciences, University of Warwick Faculty of Medicine, Coventry CV4 7AL, UK; daniel.lasserson{at}warwick.ac.uk

Abstract

Introduction Out of hours (OOHs) primary care is a critical component of the acute care system overnight and at weekends. Referrals from OOH services to hospital will add to the burden on hospital assessment in the ED and on-call specialties.

Methods We studied the variation in referral rates (to the ED and direct specialty admission) of individual clinicians working in the Oxfordshire, UK OOH service covering a population of 600 000 people. We calculated the referral probability for each clinician over a 13-month period of practice (1 December 2014 to 31 December 2015), stratifying by clinician factors and location and timing of assessment. We used Simul8 software to determine the range of hospital referrals potentially due to variation in clinician referral propensity.

Results Among the 119 835 contacts with the service, 5261 (4.4%) were sent directly to the ED and 3474 (3.7%) were admitted directly to specialties. More referrals were made to ED by primary care physicians if they did not work in the local practices (5.5% vs 3.5%, p=0.011). For clinicians with >1000 consultations, percentage of patients referred varied from 1% to 21% of consultations. Simulations where propensity to refer was made less extreme showed a difference in maximum referrals of 50 patients each week.

Conclusions There is substantial variation in clinician referral rates from OOHs primary care to the acute hospital setting. The number of patients referred could be influenced by this variation in clinician behaviour. Referral propensity should be studied including casemix adjustment to determine if interventions targeting such behaviour are effective.

  • urgent care
  • systems
  • research
  • operational

Data availability statement

All data relevant to the study are included in the article or uploaded as supplemental information. The individual patient's data used for this service evaluation are held by the NHS Trust and are not available for sharing.

https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

https://doi.org/10.1136/emermed-2020-209527

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Key messages

    What is already known on this subject

    • Primary care physicians have different referral rates to hospital from out of hours care, up to a fivefold difference between the bottom and top quintiles in referral rates.

    • The explanation for individual differences in referral rates is not clear but potential reasons include attitudes to risk as well as clinical casemix.

    • The impact of different referral rates of primary care physicians on the work of the ED and acute specialties in the out of hours period is not known.

    What this study adds

    • We used operations research methods of running simulation models to estimate the impact on referrals to hospital of different staffing models in out of hours primary care.

    • General practitioner (GP) referral rates varied between 1% and 21% of the patients they saw in the out of hours service. Removing the highest referrers from the staffing model could reduce hospital referrals by 50 patients per week.

    • Variation in referral rates by out of hours GPs may contribute to excess workload of the ED overnight and at weekends.

    Introduction

    In many developed healthcare settings, the acute care pathway includes a community-based urgent care service to complement routine or ‘office hours’ primary care. This provides an acute generalist assessment during the overnight period and at weekends for problems which do not require attendance at an ED as well as for early presentations of significant illness, where referral to hospital may be required. In the UK, out of hours (OOHs) primary care is a critical component of the urgent care pathway, undertaking 6 million clinical contacts per year.1 If patients need urgent primary medical care overnight or at a weekend in the UK, they contact a national telephone number (111) and if an assessment is required, then the call is passed to the local provider of OOH primary care. Patients will then have a telephone assessment, a face-to-face assessment at an OOH clinical base or a visiting clinician will assess the patient in their own home or care home.

    Clinicians working in OOH primary care need to manage risk in acute presenting syndromes, and this can include referral to acute hospital settings for further diagnostic testing and therapeutic interventions.2 The accurate identification of patients who require escalation of care from community to hospital settings is an essential triage function of OOH primary care,3 particularly given that 1% of patients come back to the OOH service with deterioration in their symptoms requiring hospital attendance.4

    While there has been some research on OOH general practitioners’ (GPs) decisions to escalate care from community to hospital settings, the impact of staffing strategy has not been explored. One study of an OOHs service treating a 167 000 population found a fivefold difference in referral rates from lowest to highest quintile of referrers.5 A follow-up study in the same region found that changing the organisation of OOH primary care (from individual practices to larger population-based services) had little effect on the referral rate of individual clinicians or magnitude of the differences in referral rates.6

    The extent to which referral behaviour is driven by clinician factors is currently unclear and may vary between OOH primary care and in hours primary care. One qualitative study recruiting OOH doctors highlighted that their attitudes and beliefs are likely to be strong determinants of referral rate,7 whereas quantitative surveys recruiting clinicians providing routine primary healthcare have found that casemix is more likely to be driving referral than any psychological factors at the level of the individual clinician.8

    Differences in referral rates among different OOH clinicians could adversely affect the acute care pathway in secondary care. High referrers of low-risk patients will increase the burden on acute assessment services in hospital, contributing to congestion and higher risk for all patients seen.9 Operational research can improve our understanding of the impact of this doctor-level variation in referral rates through simulation based on routinely collected healthcare data.10 Simulations using parameters derived from observed variation in routine data have been used to inform acute care design within hospitals11 12 and optimal delivery of acute stroke care at both local13 and national levels.14 However, there have been no published studies that have used operational research to determine the impact of variation in clinical referral rate from community settings to hospitals for acute illness, which is the predominant reason for consulting OOH primary care. This could help inform the optimal structure of an acute medical service at population level.

    As part of a service evaluation of referrals from the OOH primary care service, we set out to examine individual clinicians’ referral rates in a large population-based provider of OOH primary care, calculating rates from routinely collected data. We then undertook simulations to determine the impact of this variation on acute hospital referral rates.

    Methods

    The Oxfordshire OOH service, run by Oxford Health NHS Foundation Trust, provides care to a population of over 600 000 people from 18:30 to 08:00 on weekdays and 24 hour cover on weekends and bank holidays; the service has been described previously.15 An anonymised database of all patient contacts with the Oxfordshire OOHs service over 13 months from 1 December 2014 to 31 December 2015 was created from the OOH Electronic Record System used by clinicians (ADASTRA) for the purposes of evaluation of the service and identification of potential improvements to be made to the structure and processes of urgent care provided by the OOH service. The database used demographic and clinical outcome data from the ADASTRA system. We extracted data on the number of consultations undertaken by the GPs and the outcome of consultations, as well as whether the GPs undertook regular shifts with the service, practiced locally within regular primary care services and whether they were undertaking shifts as a mandatory part of the final year of training before becoming fully qualified GPs (in this situation, a qualified OOH GP would be available for supervision and advice as needed). We determined the location of assessment and any referral to the ED or direct hospital admission to specialty.

    Oxford Health NHS Foundation Trust Older People’s Directorate Clinical Audit and Effectiveness Group prospectively classified this study as a service evaluation and it was carried out under that data governance framework. In addition, University of Southampton Research Ethics Committee approved the study (ref 22990).

    Patient and public involvement

    No patient involvement.

    Statistical analysis

    Using outcomes from clinical encounters in the OOH database, we calculated counts of consultation calendar days, location and outcomes with descriptive statistics of consultation rates prepared using IBM SPSS Statistics 24 and R (www.r-project.org).

    As the decision to refer is a discrete event in time, we used discrete event simulation models using Simul8 software (https://www.simul8.com/) to represent an average week’s referrals to hospital from the OOH service, combining the codes ‘Referrals to ED’ and ‘admissions to hospital’ into one referral variable. Simulation models were constructed to explore the range of referrals under different assumptions about the levels of referral rates of individual clinicians. The referral propensity of each clinician was assumed to be their measured referral rate over a 13-month data collection period.

    We matched the current clinical model of the shift system, where each clinician will work in one of the settings of telephone consulting, face-to-face base visits or home visits. We assumed that the propensity to refer was consistent for the duration of the shift. Different assumptions were made for a total of three scenarios and each simulation model was run for 1 week, 1000 times, to build up a distribution of the variability of referrals under different assumptions. We presented average data for these runs. The assumptions behind the three models were (1) assume current staffing structure (this illustrates the impact of variation of all clinicians working in the service); (2) emergency nurse practitioners (ENPs) only (ie, no GPs) as they have a reduced referral rate overall; (3) removal of high referrers from the service (top 12% of shifts, a proportion deemed high by service managers SG and HH), which explores the impact of limiting the OOH service to clinicians who do not have very high referral rates. We assumed that arrival rates into the service were from a Poisson distribution, and that length of time waiting to be seen did not influence the decision to refer.

    Results

    There were 119 835 contacts with valid outcomes in the 13-month period with table 1 showing the number of contacts by age, gender, calendar day and location (OOH clinical base, telephone only or a home visit). The greatest number of consultations were on Saturdays and most consultations occurred as face-to-face visits at the OOH clinical base. Table 2 shows consultations by clinician factors and the majority of consultations were undertaken by sessional GPs who undertook regular work with the OOH provider. Only a small number of the total consultations (1782, 1.5%) were undertaken by trainee GPs.

    View this table:
    Table 1

    Numbers of OOHs consultations by demographic, timing, location and outcome of contact

    View this table:
    Table 2

    Number of consultations by clinician factors

    Table 3 shows the outcome of all consultations by categories of clinician. GP trainees referred more patients to hospital, either by direct admission to specialty or to the ED than qualified GPs, as a percentage of their consultations (p=0.034). GPs who did not work locally sent more patients to hospital, either as admissions to specialty or referrals to ED compared with GPs who undertake regular work in local practices (p=0.011).

    View this table:
    Table 3

    Counts and percentage of outcomes by categories of clinician

    Online supplemental table 1 shows the outcomes by location of assessment (OOH base visit, home visit or telephone only assessment) with similar patterns for impact of GP grade and regular local practice clinical work.

    Figure 1 shows the relationship between percentage of patients referred and the volume of consultations for individual clinicians which demonstrates that even above 1000 consultations within the study period, variation between 1% and 21% is seen at equivalent consultation volumes. Online supplemental figure 1 demonstrates that this pattern of variability and relationship to volume is seen across all types of clinical contact within the OOH service.

    Figure 1
    Figure 1

    Individual clinicians’ percentage of contacts referred to hospital by total number of contacts.

    Simulations

    Table 4 shows the difference in structure and parameters of the models used to simulate the impact of the variation in clinician referral rate.

    View this table:
    Table 4

    Number of patients referred to hospital in three simulation models—1000 runs of each model assuming a different staffing structure (standard mix of GPs/ENPs, ENPs only, highest referrers removed from service)

    Table 4 gives the results of the simulation model trials, of 1000 runs in each case. The total numbers referred to hospital per week are presented: the mean and the lower and upper limits of the 95% CIs are given in each case. Also reported are the minimum and maximum numbers in the trials: these represent the possible extent of arrivals at hospital that are referred from the OOH service.

    Removal of the highest referrers from the pool of clinicians available to be rostered on shifts (model iii) shows a difference in the mean number of referrals each week of 36 patients, and a reduction of the maximum weekly referrals of up to 50 patients.

    Discussion

    This evaluation of the OOH primary care service in Oxfordshire found that fewer referrals for acute assessment in hospital were made by regular providers of OOH primary care or those who worked within the local primary care practices for their routine care provision. There was substantial variation between clinicians in the percentage of contacts that were referred for urgent assessment in hospital, even among those who saw a high volume of patients. Based on insights from simulation, we conclude that variation in clinician referral rate could account for large differences in the numbers of patients who need to be assessed acutely.

    Variation in clinicians’ decision-making over change in location of care has been seen in acute settings as well as in primary care.16 A study in one acute medical unit over a 2-year period showed that the proportion of patients seen by acute physicians who are discharged home shows a similar degree of variation at individual clinician level to that seen in this study.16 Although the clinical setting is different, clinicians are being asked to make a judgement about risk related to a change in location of care—in OOH primary care the change in location is escalation to acute care, in hospital it is de-escalation to the home environment. In either setting, those who make these decisions show a high degree of variation in the propensity to escalate or de-escalate.

    This study also shows the importance of using operational research methods. The benefits of simulation modelling are, first, that the variability of numbers of patients referred to hospital can be demonstrated. Second, we can consider scenarios that could possibly take place: for example, what might happen if changes were made to the types of clinicians employed in an OOH service.

    We undertook simulations that mimic the service delivery model in OOH primary care, where clinicians work for continuous duty periods thereby fixing a referral propensity for that period. This demonstrated that there can be a substantial variation in the numbers of patients being referred to acute settings purely based on the characteristics of the doctors. Observational studies have demonstrated that high functioning OOH primary care can reduce acute hospital activity,17and this effect may be mediated through lower overall referral rates with low variation among the OOH clinician workforce in that setting.

    We found that ENPs had lower referral rates compared with GPs. There are several explanations for this finding, including ENPs seeing less complex patients, where the probability of hospital referral is lower or that ENPs have a higher threshold for hospital admission. Further research is needed to understand this observation as the reality of lower availability of GPs nationally implies that allied health professionals will be taking over clinical assessment roles in OOH services.

    There are significant pressures on acute services, particularly during winter,18 and exploring alternatives to acute assessment and treatment in hospital is a new policy focus.19 We have used operational research methods to explore how OOH primary care could be contributing to congestion in hospitals, demonstrating the value of this novel approach in identifying services that are high priority for future interventions to reduce acute hospital activity.

    Strengths and limitations

    A wide variety of models can be constructed to simulate the complex processes of care in healthcare systems.20 Our results may have been influenced by our choice of model and its assumptions. However, we sought to closely match the clinical service as far as possible and the level of complexity of the model reflects the available resources for our work. The results of this study may not generalise to other OOH primary care providers. Differences could arise from the nature of the OOH clinical workforce or different proportions of patients choosing to attend the ED as the initial contact for healthcare problems, bypassing the triage function of OOH primary care.

    We did not adjust for clinical casemix in this analysis. Given that the database is formed from patients presenting for acute OOH primary care in a relatively stable population and that clinicians work all year round, rather than in certain seasons, there is unlikely to be a large impact of casemix. Patients are seen in turn and there are no referral pathways within the OOH service, which minimises the potential for clinicians to see a higher proportion of patients at greater risk for hospital admission (eg, older patients with frailty). We did not have access to hospital admission data, so we do not know if all patients referred in the OOH database presented to the ED. Our associations with referral behaviour and clinician contracts are limited by missing data, although we did have data on over 100 000 contacts where clinician contracts were known.

    Implications for future research

    Further research should aim to use linked OOH primary care data, hospital attendance and admission data to determine the impact of variation in OOH referral rates on measures of flow in acute hospitals.

    Conclusion

    OOH primary care clinicians show high variation in their referral rates to hospitals. Simulations from routine OOH healthcare data demonstrate that this could have a significant impact on numbers of patients referred to acute hospitals.

    Data availability statement

    All data relevant to the study are included in the article or uploaded as supplemental information. The individual patient's data used for this service evaluation are held by the NHS Trust and are not available for sharing.

    Ethics statements

    Patient consent for publication

    References

    1. ↵
      1. National Audit Office
      . Out-of-hours GP services in England. London, 2014.
    2. ↵
      1. Leibowitz R,
      2. Day S,
      3. Dunt D
      . A systematic review of the effect of different models of after-hours primary medical care services on clinical outcome, medical workload, and patient and GP satisfaction. Fam Pract 2003;20:311–7.doi:10.1093/fampra/cmg313pmid:http://www.ncbi.nlm.nih.gov/pubmed/12738701
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      1. Finnikin S,
      2. Hayward G,
      3. Wilson F, et al
      . Are referrals to hospital from out-of-hours primary care associated with national early warning scores? Emerg Med J 2020;37:279–85.doi:10.1136/emermed-2019-209069pmid:http://www.ncbi.nlm.nih.gov/pubmed/31919235
      OpenUrlAbstract/FREE Full Text
    4. ↵
      1. Hayward GN,
      2. Vincent C,
      3. Lasserson DS
      . Predicting clinical deterioration after initial assessment in out-of-hours primary care: a retrospective service evaluation. Br J Gen Pract 2017;67:e78–85.doi:10.3399/bjgp16X687961pmid:http://www.ncbi.nlm.nih.gov/pubmed/27821672
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Rossdale M,
      2. Kemple T,
      3. Payne S, et al
      . An observational study of variation in GPs' out-of-hours emergency referrals. Br J Gen Pract 2007;57:152–4.pmid:http://www.ncbi.nlm.nih.gov/pubmed/17263932
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Ingram JC,
      2. Calnan MW,
      3. Greenwood RJ, et al
      . Risk taking in general practice: GP out-of-hours referrals to hospital. Br J Gen Pract 2009;59:e16–24.doi:10.3399/bjgp09X394824pmid:http://www.ncbi.nlm.nih.gov/pubmed/19105912
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Calnan M,
      2. Payne S,
      3. Kemple T, et al
      . A qualitative study exploring variations in GPs' out-of-hours referrals to hospital. Br J Gen Pract 2007;57:706–13.pmid:http://www.ncbi.nlm.nih.gov/pubmed/17761058
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Franks P,
      2. Williams GC,
      3. Zwanziger J, et al
      . Why do physicians vary so widely in their referral rates? J Gen Intern Med 2000;15:163–8.doi:10.1046/j.1525-1497.2000.04079.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/10718896
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Madsen F,
      2. Ladelund S,
      3. Linneberg A
      . High levels of bed occupancy associated with increased inpatient and thirty-day hospital mortality in Denmark. Health Aff 2014;33:1236–44.doi:10.1377/hlthaff.2013.1303pmid:http://www.ncbi.nlm.nih.gov/pubmed/25006151
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Monks T
      . Operational research as implementation science: definitions, challenges and research priorities. Implement Sci 2016;11:81.doi:10.1186/s13012-016-0444-0pmid:http://www.ncbi.nlm.nih.gov/pubmed/27268021
      OpenUrlPubMed
    11. ↵
      1. Best AM,
      2. Dixon CA,
      3. Kelton WD, et al
      . Using discrete event computer simulation to improve patient flow in a Ghanaian acute care hospital. Am J Emerg Med 2014;32:917–22.doi:10.1016/j.ajem.2014.05.012pmid:http://www.ncbi.nlm.nih.gov/pubmed/24953788
      OpenUrlCrossRefPubMed
    12. ↵
      1. Bean DM,
      2. Stringer C,
      3. Beeknoo N, et al
      . Network analysis of patient flow in two UK acute care hospitals identifies key sub-networks for A&E performance. PLoS One 2017;12:e0185912. doi:10.1371/journal.pone.0185912pmid:http://www.ncbi.nlm.nih.gov/pubmed/28968472
      OpenUrlCrossRefPubMed
    13. ↵
      1. Monks T,
      2. Worthington D,
      3. Allen M, et al
      . A modelling tool for capacity planning in acute and community stroke services. BMC Health Serv Res 2016;16:530. doi:10.1186/s12913-016-1789-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/27688152
      OpenUrlCrossRefPubMed
    14. ↵
      1. Pitt M,
      2. Monks T,
      3. Crowe S, et al
      . Systems modelling and simulation in health service design, delivery and decision making. BMJ Qual Saf 2016;25:38–45.doi:10.1136/bmjqs-2015-004430pmid:http://www.ncbi.nlm.nih.gov/pubmed/26115667
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Brettell R,
      2. Fisher R,
      3. Hunt H, et al
      . What proportion of patients at the end of life contact out-of-hours primary care? A data linkage study in Oxfordshire. BMJ Open 2018;8:e020244. doi:10.1136/bmjopen-2017-020244pmid:http://www.ncbi.nlm.nih.gov/pubmed/29712691
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Beckett DJ,
      2. Spears M,
      3. Thomson E
      . Reliable consultant level data from an acute medical unit: a powerful tool for improvement. J R Coll Physicians Edinb 2018;48:108–13.doi:10.4997/JRCPE.2018.202pmid:http://www.ncbi.nlm.nih.gov/pubmed/29992198
      OpenUrlCrossRefPubMed
    17. ↵
      1. van Uden CJT,
      2. Winkens RAG,
      3. Wesseling G, et al
      . The impact of a primary care physician cooperative on the caseload of an emergency department: the Maastricht integrated out-of-hours service. J Gen Intern Med 2005;20:612–7.doi:10.1111/j.1525-1497.2005.0091.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/16050847
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. NHS Improvement
      . NHS review of winter 2017/18, 2018.
    19. ↵
      1. Care Quality Commission
      . Under pressure. Safely managing increased demand in emergency departments, 2018.
    20. ↵
      1. Jun GT,
      2. Morris Z,
      3. Eldabi T, et al
      . Development of modelling method selection tool for health services management: from problem structuring methods to modelling and simulation methods. BMC Health Serv Res 2011;11:108. doi:10.1186/1472-6963-11-108pmid:http://www.ncbi.nlm.nih.gov/pubmed/21595946
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Handling editor Richard John Parris

    • Twitter @DanLasserson

    • Contributors DL conceived the study. HS undertook simulations and analysed data. GH contributed to study design, data collection and data interpretation. SG and HH contributed to data collection and data interpretation. DL wrote the first draft of the manuscript and all authors provided critical comment. DL is the guarantor.

    • Funding This study was not directly funded but HKS was supported by the Economic and Social Research Council (ESRC) Impact Acceleration Account (IAA) at the University of Southampton (grant award number ES/M500458/1). The work was supported by the National Institute for Health Research (NIHR) Community Healthcare MedTech and In Vitro Diagnostic Cooperative (MIC) based at Oxford Health NHS Foundation Trust and by the NIHR Applied Research Collaboration (ARC) West Midlands.

    • Disclaimer The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.