Article Text
PDF
Abstract
Objective The HOSPITAL score has been widely validated and accurately identifies high-risk patients who may mostly benefit from transition care interventions. Although this score is easy to use, it has the potential to be simplified without impacting its performance. We aimed to validate a simplified version of the HOSPITAL score for predicting patients likely to be readmitted.
Design and setting Retrospective study in 9 large hospitals across 4 countries, from January through December 2011.
Participants We included all consecutively discharged medical patients. We excluded patients who died before discharge or were transferred to another acute care facility.
Measurements The primary outcome was any 30-day potentially avoidable readmission. We simplified the score as follows: (1) ‘discharge from an oncology division’ was replaced by ‘cancer diagnosis or discharge from an oncology division’; (2) ‘any procedure’ was left out; (3) patients were categorised into two risk groups (unlikely and likely to be readmitted). The performance of the simplified HOSPITAL score was evaluated according to its overall accuracy, its discriminatory power and its calibration.
Results Thirty-day potentially avoidable readmission rate was 9.7% (n=11 307/117 065 patients discharged). Median of the simplified HOSPITAL score was 3 points (IQR 2–5). Overall accuracy was very good with a Brier score of 0.08 and discriminatory power remained good with a C-statistic of 0.69 (95% CI 0.68 to 0.69). The calibration was excellent when comparing the expected with the observed risk in the two risk categories.
Conclusions The simplified HOSPITAL score has good performance for predicting 30-day readmission. Prognostic accuracy was similar to the original version, while its use is even easier. This simplified score may provide a good alternative to the original score depending on the setting.
- Hospital medicine
- Risk management
- Transitions in care
- Adverse events, epidemiology and detection
Statistics from Altmetric.com
Introduction
Hospital readmissions are common, detrimental for patients and associated with significant costs for the healthcare system.1 ,2 In the USA in 2011–2014, more than 15% of Medicare beneficiaries aged 65 years or older were readmitted within 30 days of discharge after a medical hospitalisation.3 Preventing readmissions is therefore an important goal for the patients who would benefit from a reduction in the burden of hospitalisation, including the risks associated with each new hospitalisation, as well as for the healthcare system that would benefit from a reduction in the healthcare costs. It remains however still a challenge to prevent these undesirable events. Although it is estimated that about 73% of readmissions are not preventable, some of them may still be avoidable.4 ,5 A recent review including 42 trials on preventive interventions showed that readmission rate could be significantly reduced, with a pooled risk ratio of 0.82 (95% CI 0.73 to 0.91).6 However, the most effective interventions were also the most complex and intensive ones, addressing multiple factors related to patient context and capacity, and including among others functional status, caregiver capabilities, socioeconomic factors or potential for self-management.6 Because of the complexity and costs associated with such interventions, hospital physicians need to target them on the group of patients who are most likely to benefit, which might be patients who are at high risk of experiencing a readmission in the absence of any intervention. Unfortunately, clinicians and nurses are not good at identifying which patients are at high risk of readmission. They are actually not doing better than chance alone (C-statistic 0.50–0.58).7 Prediction models may help to better identify those high-risk patients.
The HOSPITAL score is a simple predictor model using seven clinical variables at discharge (table 1). It has been validated among nearly 200 000 patients in five different countries and showed overall good performance to predict the risk of readmission.2 ,8–12 Although most of its components are easily available, some variables of the original score would be removed to further simplify its use in real-life, and the scoring system would be simplified.
Original and simplified HOSPITAL score for 30-day potentially avoidable readmissions
In this study, we aimed to develop and internally validate a simplified version of the HOSPITAL score, which would be easier to uptake for clinicians.
Materials and methods
Study design and setting
We used the International Cohort of Avoidable REadmissions (ICARE), which is a retrospective cohort of seven university and two community hospitals in the USA, Canada, Switzerland and Israel. Details on the participating hospitals, which were all not-for-profit centres, have been described in detail elsewhere.2
The study followed the criteria from the ‘Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis’ initiative.13 The managing site (Brigham and Women's Hospital/Partners Healthcare, Boston, Massachusetts, USA) and the institutional review board of each local hospital approved the trial protocol.
Data source and participants
The ICARE included all consecutive medical patients aged ≥18 years and discharged from 1 January 2011 through 31 December 2011 at each participating hospital. Exclusion criteria were: (1) death before discharge; (2) transfer to another acute somatic or psychiatric hospital; (3) observation stay and/or length of stay of 1 day or less; (4) discharge against medical advice.
Predictor variables
HOSPITAL is the acronym for the seven predictor variables included in the original derived model (table 1):14 Haemoglobin before discharge (positive if <12 g/dL), discharge from an Oncology division, Sodium level before discharge (positive if <135 mmol/L), any International Code of Diseases (ICD)-915 or ICD-1016 coded Procedure during index hospitalisation, Index Type of admission (positive if non-elective, ie, not scheduled in advance for treatment or investigation), number of hospital Admissions within the 12 months before index admission and Length of hospital stay (positive if at least 5 days). The same definition as in the derivation study was used to collect all variables at each site.14 As all patients from two sites had been admitted at least once within the previous 12 months, we imputed 0 point instead of 1 to all patients with a single admission within this time frame. Missing values for haemoglobin (n=6907, 5.9%) or sodium (n=3980, 3.4%) were considered as normal and therefore attributed 0 point.
Two elements of the score were modified in this study. First, the variable ‘discharge from an oncology division’ was replaced by ‘discharge from an oncology division or any active cancer diagnosis’, because many hospitals do not have their patients with cancer in a specific oncology division as faced in a prospective validation study of the score,8 and because the cancer diagnosis was as highly associated with 30-readmission as oncology division.14 The diagnosis of cancer was based on the following ICD-9 codes: C00 to C96, C7A, C7B, D00 to D49. Second, ‘any procedure during index admission’ was left out, since this variable could on the one hand be less easily collected, and on the other hand was always the least significant of the model in the validation studies.2 ,14 Our assumption was that the score would be even more simple to calculate without this variable, and that the performance of the score would remain good. All patients were imputed 0 point for the variable that was left out, so that the score ranged from a minimum of 0 to a maximum of 12 points. Finally, we simplified the risk categorisation into two levels (likely or unlikely) and not three (low, intermediate and high) to allow a clear-cut decision without intermediate group of unclear significance. Similar techniques have been used in previous model simplifications.17–19
Outcome variable
The primary outcome was any 30-day potentially avoidable readmission, which was identified using an algorithm called Striving for Quality Level and Analyzing of Patient Expenses (SQLape), as it was done in the derivation study.14 Basically, this is a validated computerised algorithm based on administrative data and diagnosis codes, and commonly used since >5 years for benchmarking and comparing the different hospitals in Switzerland.20 ,21 Unavoidable readmissions include readmissions involving a new organ system unknown to be affected during the index admission, as well as foreseeable readmissions, which include transplantation, labour and delivery, chemotherapy or radiotherapy, follow-up or rehabilitation treatment, specific surgical procedures or some specific difficult to cure disorders. On the opposite, readmissions for treatment complications are classified as avoidable. For example, a pregnant woman hospitalised for pneumonia (index admission) and readmitted 2 weeks later for delivery would have an unavoidable readmission. Conversely, a patient admitted for delivery (index admission), and readmitted later for vaginal bleeding, would have a potentially avoidable readmission.
The secondary outcome was any 30-day readmission.
Statistical analysis
Baseline characteristics were presented as median (IQR), mean (SD) or frequency (%), as appropriate. If a patient was readmitted several times within the 30-day time frame, each admission following the first index one was assessed as both an index admission and a readmission. We calculated the simplified HOSPITAL score for each unit of analysis, that is, for each hospital discharge. Similar to other prediction model simplifications,17–19 we categorised the patients in two risk groups according to their score points, rather than in three groups as in the original study:22 unlikely to be readmitted if 0–4 point(s), and likely to be readmitted if 5 points or more. These categories were created for ease of interpretation, roughly corresponding to a risk of potentially avoidable readmission of more than 15% in the ‘likely’ category. We compared the prevalence of a positive score for each variable of the simplified HOSPITAL score in patients with versus those without readmission using Student's t-test.
Three different analyses were used to assess the accuracy of the simplified HOSPITAL score: (1) we calculated the Brier score to assess the overall accuracy of the scoring system, that is, how close the actual rates of readmission were to the predicted ones. A prediction model with a Brier score <0.25 is considered useful (the lower, the better).23 ,24 (2) We calculated the C-statistic of the scoring system, which represents the discriminatory power of the score, that is, the sensitivity and specificity of the model to discriminate between cases and non-cases.25 ,26 Results were presented with 95% CI. A C-statistic between 0.5 and 1 means that the score is better than random to predict the outcome (the higher the C-statistic, the better the model). (3) We assessed the calibration by fitting a logistic regression model to the data, and comparing the resultant estimates of the predicted readmission risk with the observed rates.24 In this model, we included fixed effects at the hospital level to account for variability within the different sites. Furthermore, a robust sandwich variance estimator was used to take into account repeated admissions from a single patient.27
As a rule of thumb, 10 outcomes are needed for each variable tested in a logistic regression model. With around 11 000 outcomes in our cohort population, we will have a large enough population to validate the score that contains seven variables.28
All tests were conducted as two-sided at a 0.05 level of significance. Analyses were performed with SAS Software, V.9.3 (SAS Institute).
Results
Out of 121 136 discharges from one of the nine hospitals during the study, 4701 were excluded because the patients left against medical advice or were transferred to another acute care hospital (figure 1). Among the 117 065 discharges remaining for analysis, 16 992 (14.5%) were followed by any readmission within 30 days, and 9.7% (n=11 307) by a potentially avoidable readmission.
Study flow chart. PAR, potentially avoidable readmission
Mean (SD) age of the patients at inclusion was 60.8 (18.2) years and median (IQR) length of stay during the index admission was 4 (3–7) days. Table 2 reports the baseline characteristics of the study population according to the presence or absence of a 30-day potentially avoidable readmission. Each variable of the HOSPITAL score was significantly more often positive (with a p<0.001 for each) in patients with a 30-day potentially avoidable readmission, when compared with those without. Overall, the median (IQR) simplified HOSPITAL score was 3 (2–5) points, with a range from 0 to 12 points.
Baseline characteristics of the study population according to the presence or absence of 30-day potentially avoidable readmission
Performance of the simplified HOSPITAL score
The simplified HOSPITAL score classified 70.4% (n=82 383) discharges as unlikely, and 29.6% (n=34 682) as likely to be followed by a 30-day potentially avoidable readmission (table 3). The percentage of discharges followed by a potentially avoidable readmission was 6.4% in the low-risk category and 17.3% in the high-risk category (table 3). The overall performance was very good, as reflected by a Brier score of 0.08. Discriminatory power was good also, with a C-statistic of 0.69 (95% CI 0.68 to 0.69). Figure 2 shows the receiving operating characteristic curve of the simplified HOSPITAL score. The negative predictive value of the simplified HOSPITAL score was 94%, and its specificity 73%. The calibration was excellent with predicted rates matching exactly the observed rates, as shown in table 3. When taking any 30-day readmission as outcome, the C-statistic was 0.76 (95% CI 0.76 to 0.77) and the calibration remained excellent (table 4). Overall, 29.6% of the patients were classified as high-risk, and 27.2% of them had any 30-day readmission, and 17.3% had a 30-day potentially avoidable readmission.
Observed proportions versus estimated risk of 30-day potentially avoidable readmission with the simplified HOSPITAL score
Observed proportions versus estimated risk of all 30-day readmissions with the simplified HOSPITAL score
Receiver operating characteristic curve of the simplified HOSPITAL score. The C-statistic was 0.69 (95% CI 0.68 to 0.69).
Discussion
In this study including 117 065 medical discharges, we showed that a simplified version of the HOSPITAL score can successfully predict 30-day potentially avoidable, as well as any readmissions. Among the 29.6% discharges likely to be followed by a 30-day potentially avoidable readmission, the risk of potentially avoidable readmission according to the score was 17.3%, while the observed proportion was 17.3%, showing excellent calibration. Overall accuracy was very good also, as reflected by a Brier score of 0.08. With a C-statistic of 0.69, the simplified HOSPITAL score showed similar discriminatory power as the original HOSPITAL score.2 ,14
The simplified HOSPITAL score offers two advantages in comparison to the original score. First, replacing the variable ‘discharge from an oncologic division’ by ‘cancer diagnosis or discharge from an oncology division’ enables a more extensive propagation of the use of the score in other hospital settings, including those without an oncologic division. Second, removing the variable ‘any procedure during index admission’ probably makes the score more attractive for clinicians, as capturing this variable may be difficult depending on the setting. The simplified HOSPITAL score is therefore even easier to calculate at bedside or automatically calculated in the electronic health record.
In the original derivation and validation studies, patients were classified into three categories of risk of potentially avoidable readmission, that is, low risk, intermediate risk and high risk.2 ,14 Using this three-level classification, clinicians may be unsure about how to deal with patients at intermediate risk; therefore, they may aim transition care interventions at high-risk patients only. Doing so may be detrimental for patients classified at intermediate risk but who could benefit of such interventions. A dichotomous classification may also be more convenient in clinical practice, as shown in previous simplification score studies.17–19 In this study, we therefore purposely decided to classify the patients into two categories of risk only, that is, unlikely and likely to be readmitted, and consider that this two-level scheme may be more useful to identify the patients that would most benefit of intensive transition care interventions. However, as for any prediction model, the cut-off chosen (here 5 points or more) might need to be adapted to the setting for better classification.
The HOSPITAL score presents two main advantages. First, it predicts avoidable readmissions, rather than any readmissions, which is of substantial importance, as discharge interventions should be aimed at patients that would most likely benefit, rather than for unavoidable readmissions. Second, it can be calculated before discharge, enabling targeted interventions at that time. Although one may argue that an identification of high-risk patients at admission would be more helpful to implement interventions as soon as possible, risk evolves over the course of the hospitalisation, and interventions showed to be effective were mostly performed after hospital discharge.
The aim of the HOSPITAL score was to easily identify the patients at high risk of readmission, and the variables included must be seen as good predictors, and not as an exhaustive list of modifiable risk factors. The HOSPITAL score is indeed not including factors that may be seen as very important in the risk of readmission such as socioeconomic parameters, follow-up care or home support. A good score should be easy to use, and predict with good reliability which patients are at high risk. This is what the HOSPITAL score is doing, and its good performance has been now widely validated in nearly 200 000 patients at 16 hospitals, across 5 countries and 3 continents.2 ,8–12 Because hospital readmission is particularly multifactorial, none can however expect a prediction model for readmission to reach a perfect prediction. Also, the HOSPITAL score identifies a group of patients at high risk, and not the risk at the patient level. There is however no evidence that a risk identification at the patient level is actually more useful than at a high-risk group level.
This study has some limitations. First, we included only medical patients, so that our findings may not be generalisable to surgical populations. Second, we studied the rate of readmission within 30 days after hospital discharge, a cut-off that might always be debateable, but that was chosen because it is the standard used for the Readmission Reduction Program of the Centers for Medicare and Medicaid Services in the USA, as well as in readmission policies of main European countries.29 ,30 Third, because some variables of the score may differ between different countries (eg, the variable ‘length of stay’), we are not allowed to generalise our findings to any country; however, the validation in four countries on three continents suggests a large generalisability of the score. Fourth, the algorithm used to differentiate the potentially avoidable from the non-avoidable readmissions may not provide a perfect discrimination. However, no method can argue to have a 100% sensitivity and specificity to identify the true avoidable readmissions. But the SQLape algorithm has some advantages in comparison to other methods: it has clear criteria and face validity, it is highly reproducible since based mainly on ICD codes, and it allows large database analysis. Also, we ran a sensitivity analysis with any 30-day readmission as outcome, which showed an even better discrimination power. Finally, identifying the patients at high risk of readmission with the HOSPITAL score does not give any information on which intervention should be performed, but allows to restrict the most promising and complex interventions to the patients who might benefit the most. Intervention studies targeting this population need to be done to prove the clinical impact of the use of the HOSPITAL score.
Conclusions
In conclusion, we showed that a simplified version of the HOSPITAL score does not decrease its accuracy and clinical utility, but has the potential to widen the settings in which it can be used.
References
Footnotes
Contributors JDD and JLS had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: CEA, JDD. Acquisition, analysis or interpretation of data: JDD. Drafting of the manuscript: CEA. Critical revision of the manuscript for important intellectual content: JDD, JLS, MVW, EJR, EZ, EEV, SK, JPM, TW, GSF, ADA, DA. Statistical analysis: JDD.
Funding Swiss National Science Foundation and the Swiss Foundation for Medical-Biological Scholarships (PASMP3-142734), Veterans Affairs Clinical Research Center of Excellence, Geriatric Research, Education, and Clinical Center (GRECC), National Institute on Aging of the National Institutes of Health (K23AG040157).
Competing interests JDD worked as consultant at Profility, and Homeward Health, and was supported by the Swiss National Science Foundation and the Swiss Foundation for Medical-Biological Scholarships (grant no. PASMP3-142734). EZ is a consultant and advisory board member at Earlysense, which makes monitors for patients on hospital wards, a consultant and advisory board member at Hello Doctor, which develops patient record applications, a consultant to Profility, which develops big data applications for improvement of the management of elderly populations, a founder and advisory board member at ValueScope Health, which creates financial management systems for healthcare organisations, and a founder and advisory board member at Ethos, which develops mobile health applications for patient engagement. TW has been a consultant for Novartis to provide advice on screening for cardiovascular disease. JLS has received grant funding from Sanofi-Aventis for an investigator-initiated study to design and evaluate an intensive discharge and follow-up intervention in patients with diabetes.
Ethics approval The managing site (Brigham and Women's Hospital/Partners Healthcare, Boston, Massachusetts, USA) and the institutional review board of each local hospital approved the trial protocol.
Provenance and peer review Not commissioned; externally peer reviewed.