Article Text
PDF
Statistics from Altmetric.com
Reducing 28-day or 30-day readmissions has become an important aim for healthcare services, spurred in part by the introduction of financial incentives for hospitals with high readmission rates in the USA, England, Denmark, Germany and elsewhere.1 Unfortunately, many of the most effective interventions are costly, since they are multimodal and involve several components and multiple healthcare practitioners.2 Therefore, some healthcare teams are turning to predictive models in order to identify patients at high risk for readmission and focus resource intensive readmission prevention strategies on such ‘at risk’ patients. Recent years have seen an explosion in these predictive models, which use patterns observed within large data sets to generate readmission risks for individual patients. In 2011, a systematic review found 26 models for readmissions,3 but an updated review that examined papers published up to 2015 found 68 more.4
While doubts remain about the practical value of predictive risk models (for example because it is not clear whether interventions are more effective when targeted at high-risk than low-risk patients5), it is undeniable that many models accurately predict readmission risk. Among the 14 published models that target all unplanned readmissions (rather than readmissions for specific patient groups), the ‘C statistic’ ranges from 0.55 to 0.80, meaning that, when presented with two patients, these models correctly identify the higher risk individual between 55% and 80% of the time. As a benchmark, consider one study6 that asked practitioners to estimate the 30-day readmission risk for patients discharged from their tertiary medical centre in 2008. Staff physicians, residents and interns correctly predicted patients who would return to hospital within 30 days with a C statistic of around 0.58 (considerably below the typical target for acceptable discrimination of at least 0.7). Nurses and case managers performed little better than chance (with C statistics of 0.55 and 0.50, respectively) in predicting readmissions at the time of discharge.6 It is possible that the predictions of healthcare practitioners have improved since 2008, due to the many insights since published in the literature regarding the causes of readmissions.7 Nonetheless, it seems likely that some predictive models outperform clinicians when it comes to discriminating between patients at high and low readmission risks.
Now that the technical feasibility of predictive modelling has been demonstrated, it is timely to ask where next. In this editorial, we argue that the priority lies with developing logic models that link the outputs from these models to the decisions practitioners need to make regarding the care of individual patients. This contrasts with the apparent direction of the research field, which sometimes seems more intent on the pursuit of increasingly complex analytical methods. Although further innovation in analytical methods is possible (eg, using neural networks, decision trees or random forests,8 or by incorporating information from electronic health records)9, it is striking that many of the most well-validated (and perhaps therefore, the most useful) models have adopted comparatively simple approaches. For example, the HOSPITAL score is a weighted summation of just seven variables,10 and produces C statistics over 0.70 when applied to international data11 12 (in this issue of BMJ Quality and Safety, Aubert and colleagues have managed to simplify that model still further, while retaining a C statistic at around the same level13). Another model, LACE, uses just four variables (Length of stay, Acuity of the admission, Charlson Comorbidity score and number of previous Emergency department visits), and produced a C statistic of 0.68 when validated on administrative data from Canada,14 though lower in some international data sets.12 15
Although more complicated models might be appealing (especially when marketed using labels like ‘machine learning’ or ‘artificial intelligence’), surely the greater priority consists of testing and developing approaches to use predictive models in ways that improve outcomes for patients. This will require linking predictive models to actionable opportunities for improving care. Such linkages will most likely be identified through close collaboration between analytical teams, healthcare practitioners and patients. In outlining our thinking below, we have drawn on lessons learnt from earlier predictive modelling efforts, many of which were focused on admissions rather than readmissions.16 17
The case for collaboration between analytical teams and healthcare practitioners
Even setting aside concerns around the digital platform on which predictive models may be implemented, the algorithms themselves constitute a form of ‘analytical tech.’ As with any technology, their impact will depend on how people interact with them as part of a ‘sociotechnical system’.18 And, healthcare teams will not take up these models overnight, but only if there is sustained engagement with users to understand the purpose of the models.19 Therefore, it is quite alarming that, with so many predictive models being developed, so few studies have examined practitioner attitudes to using them.
In a rare example, Porter and colleagues sought the views of primary care practitioners in Wales regarding predictive models.20 Their focus was on admissions, rather than readmissions, and how the models might be used to identify patients for case management interventions. Practitioners could see possibilities to use the models to offer care on a more proactive and orderly basis, but only if interventions existed to reduce the admission risk of the identified patients, and the surrounding support services were available to implement these interventions effectively, while managing the potential for extra demand for healthcare. These findings are not surprising, and are likely to hold true for readmissions as well as admissions. They underline the need to consider how predictive models will function as part of the broader approach to care delivery. Without a clear pathway for using these models to improve care, they are unlikely to be brought into use.
Another issue is that being at ‘high risk’ is not the only requirement for enrolment into intensified care programmes: the patient must also stand to benefit by the changes being considered, or in the phrase commonly used in the predictive modelling literature, must be ‘impactible.’21 For instance, a predictive model could easily identify a patient with poorly controlled diabetes, advanced heart failure and alcohol abuse as a high risk for readmission. But, the patient’s repeated refusal to pursue alcoholism counselling or treatment might make him not very impactible in terms of interventions to reduce the risk of readmission. While predictive models are designed to assess risk, the assessment of impactibility requires consideration of factors such as the willingness and ability of patients to participate in programmes that can improve their outcomes. Such an assessment is likely to require clinical experience.
The distinction between predictive risk and impacibility might explain why practitioners tend to identify quite different patients for intervention than predictive risk models. One study, again focused on admissions rather than readmissions, asked 14 primary care physicians in Germany to identify patients for a future case management programme.22 Of the 233 patients thus identified, only 30 were in the top decile of risk of future hospitalisation according to a predictive model.23 Not surprisingly, the 311 patients identified by the predictive model were more likely to go on to experience a hospital admission than those identified by the primary care physicians (49% vs 28%). Yet, the patients identified by the physicians had characteristics that, at least intuitively, suggest greater impactibility. For example, they showed a trend towards increasing admissions over time, whereas those patients identified by the predictive model had reducing rates of admission (although sustained at a higher level). Also, the physicians identified patients with lower 1-year mortality rates than the predictive model (2% vs 10%), which might indicate greater scope to intervene with patients outside of a palliative care environment. Finally, the physicians were more likely to identify patients with a history of participating in disease management programmes,24 and who thus might be more amenable to participating in future programmes.
These considerations suggest that the interaction of predictive models and clinicians might produce more effective and equitable decision making than either alone.25 One of the strengths of predictive models is that they produce objective and consistent judgements regarding readmission risk, whereas clinical judgement can be affected by personal attitudes or attentiveness. Predictive risk models can also be operationalised across whole populations, and might therefore identify need that would otherwise be missed by clinical teams (eg, among more socioeconomically deprived neighbourhoods or groups with inadequate primary care). On the other hand, clinicians have access to a much wider range of information regarding patients than predictive risk models, which is essential to judge impactibility. A literature is emerging regarding the interaction of analytical technology and humans—a phenomenon that some researchers have called ‘cyborg practices.’26
The need to engage more systematically with patients
There are compelling arguments to involve patients in the development of predictive risk models for readmissions. For example, many of the modifiable risk factors for readmissions relate to the behaviour of patients,27 and indeed the effectiveness of interventions to reduce readmissions is in part related to whether or not they promote self care.2 There is a danger that, without patient involvement, these risk factors might not be incorporated into predictive risk models, or receive sufficient attention when designing the surrounding approach to care delivery. The problem is in part related to the nature of existing healthcare data sets.
Analysts developing predictive models often rely onadministrative databases and collections of electronic medical records, and rightly refer to the benefits of these data sets in providing a longitudinal, population-wide resource. Yet, the increasingly easy access to these data sets may have reduced the incentives on analysts to engage with patients directly to collect data on predictors of readmissions. It is interesting that the earliest predictive models relied on survey data obtained from patients, and so included factors not commonly recorded in healthcare data sets, such as self-rated health and the availability of informal care support.16 More recent models have tended to ignore these, potentially to their detriment.
Raven and colleagues explored a model that had previously shown good performance at predicting 12-month readmissions, based on data on prior service utilisation and diagnoses from administrative data.28 When they subsequently interviewed high-risk patients,29 they uncovered high rates of social and economic risk factors, with 60% being homeless or precariously housed with family of friends, 52% living alone, 64% having two or fewer friends or relatives with which whom to discuss important issues, 70% experiencing moderate or high-risk substance abuse or dependence, and 76% having levels of anxiety or depression exceeding the general population. Of course, the findings might not be representative of every population, since the study was conducted at Bellevue Hospital Center, which operates within an urban and medically underserved area. Yet, they are consistent with a wider literature that points to the relation between readmission rates and the resources available to patients.2 30
While the model tested by Raven and colleagues performed well, it did not include data on the social and economic risk factors that were uncovered by the subsequent interviews. Data on these factors might be used to improve the predictions still further, or to spot opportunities to improve care. Yet, very few studies have examined whether data collected prospectively from patients add to the predictive power of existing risk models for readmissions. Mixon and colleagues examined an 11-item measure that assesses how prepared patients feel when leaving the hospital (B-HOSPITAL).31 They concluded that it did not add meaningfully to the LACE index when predicting 30-day readmissions for patients with cardiovascular diseases, but might still help to direct care transition quality improvement efforts. Other metrics might be tested, for example patient activation measures,32 and should ideally be explored within structured, mixed-methods studies.33
It seems a missed opportunity to not collect data from patients, especially when predictive models that are derived using large databases are later being simplified to enable them to be implemented at the bedside.13 There may be ample opportunity to collect data from many patients during the discharge planning process. A more general point is that many healthcare systems rely predominately on practitioners and administrative staff to collect data, and generally lack ways to collect data on an ongoing basis from service users.34 Techniques are available (eg, surveys and e-health apps) but are most effectively pursued within a context that engages healthcare teams and service users in service development. This is because clear logic models are needed regarding how the data will be used to improve care.35
Conclusions
Collaborating with patients and practitioners when developing predictive risk models will not by itself solve some of the other conundrums in this area, such as which interventions should be delivered for which risk groups, or how those interventions should be resourced, evaluated and improved.5 Yet, the first step in any quality improvement project consists of understanding the nature of the problem at hand, and this understanding requires close working between analytical teams, healthcare practitioners and patients. The predictive modelling enterprise would benefit enormously from such collaboration because the real goal of this activity lies not in predicting the risk of readmission, but in identifying patients at risk for preventable readmissions and ‘impactible’ by available interventions.
References
Footnotes
Competing interests None declared.
Provenance and peer review Commissioned; internally peer reviewed.
Correction notice This paper has been amended since it was published Online First. Owing to a scripting error, some of the publisher names in the references were replaced with ‘BMJ Publishing Group’. This only affected the full text version, not the PDF. We have since corrected these errors and the correct publishers have been inserted into the references.