• information technology
• decision support
• computerised
• implementation science
• qualitative research
• trigger tools

Palliative care is associated with improved patient-centred and caregiver-centred outcomes, higher-quality end-of-life care, and decreased healthcare use among patients with serious illness.1–3 The Centre to Advance Palliative Care has established a set of recommended clinical criteria (or ‘triggers’), including a projected survival of less than 1 year,4 to help clinicians identify patients likely to benefit from palliative care. Nevertheless, referrals often occur within the last 3 months of life5 due in part to clinician overestimation of prognosis.6 A growing number of automated predictive models leverage vast data in the electronic medical record (EMR) to accurately predict short-term mortality risk in real time and can be paired with systems to prompt clinicians to refer to palliative care.7–12 These models hold great promise to overcome the many clinician-level and system-level barriers to improving access to timely palliative care. First, mortality risk prediction algorithms have been shown to outperform clinician prognostic assessment, and clinician–machine collaboration may even outperform both.13 Second, algorithm-based ‘nudges’ that systematically provide prognostic information could address many cognitive biases, including status quo bias and optimism bias,14 15 that make clinicians less apt to identify patients who may benefit from palliative care. Indeed, such models have been shown to improve the frequency of palliative care delivery and patient outcomes in the hospital and clinic settings.9 16 17 With that said, successful implementation of automated prognostic models into routine clinical care at scale requires clinician and patient engagement and support.

In this issue of BMJ Quality & Safety, Saunders and colleagues report on the acceptability of using the EMR-based Modified Hospitalised-Patient One-Year Mortality Risk (mHOMR) score to alert clinicians to individual patients with a >21% risk of dying within 12 months. The goal of the clinician notification of an elevated risk score was to prompt clinicians to consider palliative care referral.18 In a previously reported feasibility study among 400 hospitalised patients, use of the mHOMR alert was associated with increased rates of goals of care discussions and palliative care consultation in comparison to the preimplementation baseline (34% vs 18%, respectively).19 In the present study, the authors conducted qualitative interviews pre-mHOMR and post-mHOMR implementation among 64 stakeholders, including patients identified at high risk by the mHOMR algorithm, their caregivers, staff and physicians. Thirty-five (55%) participants agreed that the mHOMR tool was acceptable; 14 (22%) were unsure or did not agree; and 15 (23%) did not respond. Participants identified many potential benefits of the programme, citing the advantages of an automated approach to facilitate and justify clinical decision making. Participants also acknowledged possible barriers, particularly ‘situational challenges’ such as the content, timing and mechanism of provider notification. Additional logistical concerns included alert fatigue, potential redundancy, uncertainty regarding next steps and a worry that certain therapeutic options could be withheld from flagged patients. The authors concluded that clinicians and patients found the automated prognostic trigger to be an acceptable addition to usual clinical care.

Saunders et al’s work adds to our understanding of critical perceptions regarding end users’ acceptability of automated prognostic triggers in routine clinical care. The findings from this study align with prior evidence suggesting that clinicians recognise the value of automated, algorithm-based approaches to improve serious illness care. For example, in a qualitative study of clinicians by Hallen et al, prognostic models confirmed clinicians’ gestalt and served as a tool to help communicate prognosis to patients.20 Clinicians described prognostic models as a tool to facilitate interclinician disagreements, mitigate medicolegal risk, and overcome the tendency to ignore or overestimate prognosis.20 Clinicians also reported that EMR-generated lists of high-risk patients improved their ability to identify potential palliative care beneficiaries in a mixed-methods study by Mason et al.21 In a single-centre pilot study, we similarly found that most clinicians believed that using an EMR-based prognostic model to encourage inpatient palliative care consultation was acceptable.9 However, in the Saunders et al study, as in prior similar work, clinicians highlighted the importance of delivering notifications without causing excess provider workload, redundancy or alert fatigue.16 18 21 Clinicians also raised concerns regarding the accuracy of the prognostic information and the potential for negative effects on patients due to common misperceptions about palliative care being equivalent to hospice.18 20 21 Ultimately, Saunders et al’s work complements and builds on existing literature, demonstrating a general perception that integration of automated prognostic models into routine clinical care could be beneficial and acceptable.

Important gaps remain in this literature which were not addressed by the Saunders et al study. For example, there is a need to capture more diverse clinician and patient perspectives, and there was no information provided about the sociodemographic or clinical characteristics of the study participants. Additionally, important themes found in prior studies were not identified in this study. For example, two prior studies of clinicians’ perspectives on automated prognostic triggers for palliative care revealed concerns that prognosis alone may not be a sufficient surrogate indicator of actual palliative care need, or may inadvertently engender clinician overconfidence in an individual patient’s prognosis.9 21 The brevity of the interviews in Saunders et al’s study (mean: 12 min) could suggest all relevant themes may not have emerged in the data analysis. Additionally, while the inclusion of patient and caregiver perceptions is an important addition, limited information is provided about their perspectives and whether certain themes differed among the stakeholders. In the study from Mason et al, themes unique to patients and caregivers were identified, such as hesitancy due to a lack of understanding of palliative care, a preference to ‘focus on the present’, and a worry that a clinician would not have the time to adequately address advanced care planning or palliative care during their visit.21 Healthcare systems should therefore be prepared to consider their unique workflows, patients and staff prior to implementing one of these programmes.

Achieving stakeholder acceptability prior to widespread implementation is essential. An intervention should ideally undergo multiple cycles of optimisation with ongoing appraisal of patient and clinician perspectives prior to wide-scale implementation.22 23 Additionally, it is unclear whether clinicians’ acceptability of the intervention in one setting will generalise to other inpatient health settings. For instance, Saunders et al found that some providers were leery about the use of mHOMR due the need to balance the patient’s acute needs that brought them to the hospital with their long-term priorities that may be better served in the outpatient setting.18 Clinical workflows, patient acuity and patient–provider relationships are markedly different between the inpatient and outpatient settings, suggesting Saunders et al’s findings cannot be extrapolated to outpatient care. This is particularly relevant as many ‘off-the-shelf’ prognostic algorithms are now commercially available that, while accurate, may not be as familiar or acceptable to clinicians as a homegrown model. Therefore, while Saunders et al’s work is a great addition to the field, additional assessments are needed across different healthcare environments and varying clinical and demographic cohorts to demonstrate that this approach is acceptable in other health settings. It is likely that multiple implementation strategies will be needed to successfully adapt automated prognostic models across a range of clinical settings.

Thoughtful consideration of the many forces that alter clinical decision making will also be critical for downstream success of these interventions. Suboptimal clinical decision making is often a result of systemic biases, such as status quo and optimism bias, which result in clinician resistance to change current practice and a belief that their patients are less prone to negative outcomes.14 15 Intentional application of targeted behavioural economics principles will help ensure that the use of prognostic triggers to improve palliative care effectively changes clinical behaviour.24 For example, using an ‘opt-out’ approach for palliative care referral may make the optimal choice the path of least resistance, increasing uptake among clinicians.16 These approaches will need to be balanced against rising clinician alert fatigue25 and resource constraints.

Given the implementation challenges that accompany an intervention using prognostic triggers, hybrid effectiveness trials that test both clinical effectiveness and implementation outcomes offer one strategy to advance the integration of automated prognostic models.26 Implementation outcomes are typically based on a framework which provides a systematic way to develop, manage and evaluate interventions. For example, Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) is a framework that measures the impact of a programme based on five factors: reach, effectiveness, adoption, implementation and maintenance.27 Due to their pragmatic approach, hybrid trials frequently include heterogenous samples and clinical settings that optimise external validity and generalisability.26 28 They can be designed to primarily test the effects of a clinical interventions while observing and gathering information on implementation outcomes (type I), for equal evaluation of both the clinical intervention and implementation strategies (type II), or to primarily assess implementation outcomes while collecting effectiveness data (type III).26 29 For example, Beidas et al used a type I hybrid effectiveness–implementation trial design to test the effectiveness of an exercise intervention for breast cancer. This study not only evaluated the effectiveness of the intervention but also identified multiple significant implementation barriers such as cost, referral logistics and patient selection challenges which informed their subsequent dissemination efforts.30 Prospective, randomised, hybrid effectiveness–implementation designs focusing on other key implementation outcomes are a logical and necessary next step in advancing the field. In total, the work by Saunders et al demonstrates the potential acceptability of an automated prognostic model to improve the timeliness of palliative care, setting the stage for further work to optimise and implement these programmes into real-world clinical care.