Article Text
PDF
Abstract
Objectives To develop and validate a tool to predict the risk of an older adult experiencing medication-related harm (MRH) requiring healthcare use following hospital discharge.
Design, setting, participants Multicentre, prospective cohort study recruiting older adults (≥65 years) discharged from five UK teaching hospitals between 2013 and 2015.
Primary outcome measure Participants were followed up for 8 weeks in the community by senior pharmacists to identify MRH (adverse drug reactions, harm from non-adherence, harm from medication error). Three data sources provided MRH and healthcare use information: hospital readmissions, primary care use, participant telephone interview. Candidate variables for prognostic modelling were selected using two systematic reviews, the views of patients with MRH and an expert panel of clinicians. Multivariable logistic regression with backward elimination, based on the Akaike Information Criterion, was used to develop the PRIME tool. The tool was internally validated.
Results 1116 out of 1280 recruited participants completed follow-up (87%). Uncertain MRH cases (‘possible’ and ‘probable’) were excluded, leaving a tool derivation cohort of 818. 119 (15%) participants experienced ‘definite’ MRH requiring healthcare use and 699 participants did not. Modelling resulted in a prediction tool with eight variables measured at hospital discharge: age, gender, antiplatelet drug, sodium level, antidiabetic drug, past adverse drug reaction, number of medicines, living alone. The tool’s discrimination C-statistic was 0.69 (0.66 after validation) and showed good calibration. Decision curve analysis demonstrated the potential value of the tool to guide clinical decision making compared with alternative approaches.
Conclusions The PRIME tool could be used to identify older patients at high risk of MRH requiring healthcare use following hospital discharge. Prior to clinical use we recommend the tool’s evaluation in other settings.
- Medication harm
- older adults
- healthcare use
- hospital discharge
- risk prediction
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Background
Reducing the burden of serious and avoidable medication-related harm (MRH) by 50% by 2022 is WHO’s third global patient safety challenge.1 Transitions of care, particularly following hospital discharge,2 present a heightened risk and addressing this is a priority.3 Medication-related harm includes harm from adverse drug reactions (ADR), non-adherence and medication errors.2
Older adults are especially vulnerable to experience MRH due to multimorbidity and polypharmacy,4 age-related changes in pharmacokinetics and pharmacodynamics5 and adherence problems.6 Patients and carers identify the care transition around hospital discharge as a unique, high-risk period for the occurrence of MRH.7–9 Contributory factors include deconditioning during hospitalisation, ongoing recovery from acute illness,10 inaccuracies in medicines reconciliation,11 insufficient patient education of medicines use12 and poor communication between secondary and primary care.13 14 A systematic review of MRH in older adults found that between 17% and 51% of patients experience MRH within 30 days of hospital discharge.2 In the UK, 28% of older adults (≥65 years) use health services due to MRH within the 8 weeks following hospital discharge, at a cost of £400 million (US$530 million) to the National Health Service (NHS) annually.15
Current strategies to address MRH during transitions of care include medicines review incorporating a patient-centred approach to deprescribing16 17 and the improvement of information transfer through medicines reconciliation.18 Risk stratification using prediction tools is recognised as one solution to enhance the cost-effectiveness of interventions by identifying patients likely to derive greatest benefit.19 20 Individual risk prediction can inform clinical decision making and support patient participation in their healthcare. Risk prediction tools such as the Framingham21 or QRISK22 tools for estimating cardiovascular risk have transformed the provision of healthcare.
Currently, no tools are available to target intervention to patients at high risk of MRH in the community following hospital discharge.23 Relying on clinical judgement alone is poorly predictive.24 A key recommendation in a position statement from the International Group for Reducing Inappropriate Medication Use and Polypharmacy is to ‘develop tools that can aid the detection and management of drug adverse effects’.25
The aim of our study was to develop and validate a risk prediction tool to identify older patients at high risk of MRH requiring healthcare use within 8 weeks following hospital discharge.
Methods
Design, setting and participants
The risk prediction tool reported in this paper was developed using data from a multicentre, prospective cohort study; Prospective study to develop a model to stratify the RIsk of Medication-related harm in hospitalised Elderly patients (PRIME).15 26 Detailed methods are in the published protocol.26 This study is reported in accordance with the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement.27
Research nurses recruited inpatients (age ≥65 years) close to discharge from medical wards, between 2013 and 2015, in five NHS teaching hospitals in England. Patients were excluded if they were terminally ill, if they lacked capacity with no nominated consultee or if they were transferred to other acute healthcare units.
Baseline data collection
Research nurses collected baseline information from consenting patients using medical records and patient interview, including demographic, clinical and social data.
Outcome definition
The risk prediction model was developed to predict MRH requiring healthcare within 8 weeks of hospital discharge. MRH was defined as an ADR or harm arising from non-adherence. Non-adherence was defined as failure to use medicines as the prescriber intended. Harm arising from medication error was included where reported. Intentional overdose was excluded. Asymptomatic derangements in blood markers were not included as MRH. This is a modified version of the definition by Strand et al.28
Healthcare use included primary, secondary or tertiary consultations related to MRH. Senior, research pharmacists followed participants for 8 weeks to determine MRH occurrence. An 8-week observation period was chosen as previous research outside of the UK had found this is a reasonable timeframe to capture most postdischarge MRH.29–31 Classification of MRH events was completed using the validated Naranjo algorithm as ‘possible’, ‘probable’ or ‘definite’.32–35 Where an event was not suspected, this was classified as ‘doubtful’. To eliminate uncertainty and strengthen the external validity of the model, any events that were possible or probable were excluded. Only events that required healthcare use were included as an objective threshold for MRH severity.
MRH and healthcare use assessment
Pharmacists identified MRH and healthcare use (accident and emergency department attendance, hospital admission, outpatient clinics, general practitioner (GP) in-person or telephone consultations and out-of-hours care) from three sources: (1) participant/carer telephone interview at 8 weeks using a structured questionnaire, (2) GP records and (3) prospective review of hospital readmissions with the admitting medical consultant.
The Naranjo Algorithm35 was used to assess ADR causality in conjunction with the British National Formulary and Summary of Product Characteristics. Clinical judgement supported by a validated questionnaire36 and prescription order data were used to determine if non-adherence had contributed to MRH.
Two senior study pharmacists provided case-based training to all other pharmacists. Additionally, cross-site case discussions were held to ensure standardisation of MRH assessments.
An end point committee independent from data collection, consisting of a professor of geriatrics, professor of clinical pharmacy and two consultant geriatricians reviewed, scrutinised and verified MRH cases.
Selection of candidate predictors
Candidate variables for the PRIME tool were prospectively selected by (1) two systematic reviews exploring previous MRH risk prediction tools, and, the epidemiology of MRH postdischarge,2 23 (2) three round-table expert meetings to identify candidate predictors, prior to any data analysis, based on clinical relevance and practicality of routine measurement (including two professors of geriatrics, two consultant geriatricians, one professor of pharmacy, one hospital chief pharmacist, one pharmacy research fellow and one geriatrics research fellow), (3) a qualitative study to identify MRH risk factors from the patient and carer perspective.37 At this initial stage no upper limit was placed on the number of candidate variables.26 38 All candidate variables were selected based on clinical relevance and not using univariable analysis to avoid the possibility of variable inclusion by statistical chance.39 40 Once the number of MRH events in the study was known, the number of variables was reduced for model development recognising 10 events per variable by Peduzzi et al as a rough indicator.38 However, the use of this ‘rule of thumb’ is not universally advocated.41 42
Sample size
The study was adequately powered based on a calculation using the nomogram designed by Carley et al43 and based on the work of Buderer et al.44 A sample size calculation was performed to achieve a sensitivity of 80% with a 95% CI width of 5% and based on an MRH prevalence rate of 30% (see online supplementary file for additional detail).
Supplemental material
Missing data
Under a missing at random assumption, we imputed data on all variables with missing data; estimated glomerular filtration rate, hand grip strength, past ADR and living alone postdischarge. These values were replaced using Multiple Imputation by Chained Equations.45 For each variable 10 imputed datasets were created and Rubin’s rule was used to obtain an overall estimate.45 See online supplementary file for additional detail.
Statistical analysis
All statistical analyses were conducted in Stata V.14 (StataCorp, College Station, Texas, USA).
Model development
The primary outcome was whether a person experienced MRH requiring healthcare use; a binary variable. Backward elimination, based on the Akaike Information Criterion (AIC), excluded variables that were not contributing sufficient predictive information in the multivariable logistic regression model46 (the AIC is a predictive metric that equates to the elimination of variables with p≥0.157). The AIC was chosen as the selection procedure as it balances the risk of overfitting a model to the derivation data and introducing misspecification error from trimming potentially meaningful variables, while penalising the inclusion of redundant variables contributing little predictive information to generate a parsimonious model.47 Two variables (age and sodium level) were centred on their mean values to make the interpretation of the intercept easier and meaningful. In the model equation, the intercept can be interpreted as the predicted risk of MRH if all the continuous explanatory variables are set to 0 (and the categorical variables held at their baseline values), a value which is infeasible for continuous variables such as age and sodium level. Therefore by centring these two variables, the intercept in the model equation would indicate the predicted risk of MRH with plausible values for age and sodium level. Two-way interactions between pairs of variables were tested and no significant interactions were found between them in their relationship with MRH.
A final model of eight predictors was obtained after backward elimination and a risk equation for predicting the log odds of MRH was formed by summing the products of the estimated β-coefficients (obtained from fitting the multivariable logistic regression model) and their corresponding observed predictors, plus the intercept.
All the predictors were assessed for collinearity by calculating the variance inflation factor (VIF) to identify whether variances of the estimated coefficients were inflated.48
Evaluating model performance
The performance of the model was evaluated by its discrimination (area under the receiver operating characteristic curve) and calibration (calibration slope and Hosmer-Lemeshow goodness-of-fit statistic).49
Bootstrap correction of model optimism and validation
Bootstrap is a model validation approach that is recommended when an external cohort of patients is unavailable to estimate the performance of the prediction model.50–53 Bootstrap is a resampling method that is used to randomly generate data (data for subsets of patients) from the original sample with replacement (patients can be selected multiple times). The bootstrap sample is the same size as the original sample (n=818). One hundred samples of the derivation data were bootstrapped, and, in each sample a prediction model was developed and used to compute an estimate of model optimism (see online supplementary file for additional detail).
Assessment of potential clinical usefulness
A net benefit approach using decision curve analysis was conducted to explore the potential clinical utility of the PRIME tool. This type of analysis measures the benefits and harms (in terms of true positives and false positives) of using the tool to guide intervention,54 that is, in the current context, intervention to avoid MRH. A decision analytic curve can compare the net benefit of using this new approach with existing approaches to guide intervention, for example, polypharmacy,16 17 or an approach of treating all people of a particular risk threshold, or alternatively treating none. The ‘threshold’ risk at which a clinician would decide to intervene is subjective and should incorporate patient preferences. Furthermore, the threshold will depend on the time and cost resource, invasiveness, effectiveness and harms of the intervention.54 55 However, a reasonable range of threshold probabilities in the context of intervening on the risk of serious MRH was considered to be 5%–25%. This range reflects expert consensus opinion of the multidisciplinary PRIME study coordinating team. The assumption here that it would be surprising if a patient had <5% risk of serious MRH and a decision was made to intervene, that is, an acceptance of intervening on 19 patients unnecessarily to avoid one patient experiencing MRH. Likewise, if a patient has >25% (one in four) chance of experiencing serious MRH it would be a surprising decision to take no action to mitigate this risk.
Results
Participant characteristics
A total of 1280 older adults were recruited at hospital discharge for 8-week follow-up. Seventeen participants died without follow-up (1.3%), 147 participants (11.5%) were lost to follow-up because they were uncontactable and GP records were unavailable (see online supplementary file for further detail and participant flow chart). One hundred nineteen participants experienced ‘definite’ MRH requiring healthcare use within 8 weeks following hospital discharge (an incidence of 107 participants per 1000 discharged). The model derivation cohort included 818 participants with either ‘definite’ MRH requiring healthcare use or no MRH requiring healthcare use (table 1). The ‘possible’ and ‘probable’ cases of MRH, as well as ‘definite’ cases that did not require healthcare use were excluded.
Baseline sample characteristics
Model development and performance
Twenty-five candidate variables were initially identified based on clinical judgement, existing literature and qualitative work with patients and carers. Three of the 25 variables (albumin level, C reactive protein, white cell count) were excluded due to substantial missing data (>20%). Two variables (change in accommodation after discharge and hepatic impairment) were excluded due to insufficient prevalence in the cohort (<10%). One variable (cardiovascular drug on discharge) was excluded due to saturated prevalence in the cohort (>80%). The remaining 19 variables were taken forward for potential inclusion in the final multivariable analysis, and 12 predictor combinations were trialled to obtain a parsimonious model with optimal performance and stability (broadly adhering to the 10 events-per-variable ratio rule of thumb38 to reduce model performance bias56). Models were examined for evidence of multicollinearity, and where demonstrated the variable contributing the least predictive value was excluded. The iterative procedure resulted in the exclusion of a further seven variables (regular falls, Barthel score, addition of a new drug, opiate drug, anticoagulant drug, abbreviated mental test score, depression on screening). The 12 predictors included in a final multivariable model to generate the PRIME tool are shown in table 2 (see online supplementary file for summary of excluded variables).
Selected candidate predictors to derive the risk prediction model
Backward elimination using the AIC (equating to p=0.157)46 resulted in a final risk prediction model of eight variables: age, gender (female=0), antiplatelet drug (antiplatelet on discharge=1), sodium level, antidiabetic drug (antidiabetic on discharge=1), history of ADR (past ADR=1), number of discharge medicines and living alone (living alone postdischarge=1). The final model equation along with case examples to illustrate use of the prediction tool is shown in figure 1.
Prime tool to calculate patient risk of experiencing MRH requiring healthcare use within 8 weeks following hospital discharge. ADR, adverse drug reaction; GP, general practitioner; MRH, medication-related harm.
The linear relationship between continuous predictors (modelled on their continuous scales) and the log-odds of the outcome variable was assessed by the use of multivariable fractional polynomials. A Wald test showed that there was a linear relationship between the continuous variables and the log-odds of the outcome. No multicollinearity was demonstrated in this model (mean VIF=1.07). The apparent discrimination of the model was AUC 0.69 (95% CI 0.64 to 0.74) (figure 2). For comparison, the number of medicines alone discriminated between higher and lower risk patients with AUC 0.61 (95% CI 0.55 to 0.66). A sensitivity analysis found a poorer model discrimination if ‘possible’ and ‘probable’ MRH were categorised with ‘definite’ MRH (C-statistic 0.64, 95% CI 0.60 to 0.67), or if ‘probable’ events were grouped with ‘definite’ MRH (C-statistic 0.65, 95% CI 0.61 to 0.69).
Prime prediction tool compared with number of medicines alone to discriminate patient risk of medication harm. ROC, receiver operating characteristic.
The optimism of the model’s discriminatory performance was 0.028, calculated as the average difference between model discrimination in the bootstrap sample and the discrimination of the bootstrap model in the original sample over 100 bootstrap iterations. The optimism adjusted model discrimination is AUC 0.66 (95% CI 0.61 to 0.71).
The calibration, that is, the level of agreement between predicted and observed risk probabilities was good (see online supplementary file for calibration plot). The Hosmer-Lemeshow goodness-of-fit statistic was 5.47, with 8 df, p=0.71 indicating no evidence of statistically significant difference between the observed and expected values. A uniform shrinkage factor of 0.85, derived from the average of the calibration slopes of the bootstrap iterations, was applied to predictor coefficients to adjust the risk prediction model for optimism. Figure 3 shows the results of the decision curve analysis, which demonstrates the superior net benefit that use of the PRIME tool offers compared with alternative approaches of ‘treat all’, ‘treat none’ or using a binary polypharmacy model to aid clinical decision making over a range of threshold probabilities of approximately 5%–40%.
Decision curve comparing the net benefit of alternative models for clinical decision making.
Discussion
The PRIME tool is the first validated tool to predict the absolute risk of an older person experiencing medication harm in the postdischarge period; the risk of MRH in this transition period is approximately three times greater than the inpatient or ambulatory setting.2 15 33 57 Major strengths of the PRIME tool are that: (1) it was developed using multicentre prospective data identifying a robust outcome that has relevance to patients, practitioners and policy makers (ie, definite MRH requiring healthcare use), (2) candidate predictors were clinically derived with patient and carer views considered, (3) the variables in the tool are routinely collected inpatient data, (4) the tool shows potentially superior clinical utility compared with some current risk stratification approaches based on a decision curve analysis.
This tool could support the WHO’s campaign to reduce MRH during transitions of care1 by flagging high-risk patients to target medicines optimisation interventions, and enabling clinicians to more accurately evaluate the risks of polypharmacy for individual patients. Furthermore, the personalised information from the tool could inform more individualised discussions around medication safety and enhance patient autonomy.58
Comparison with other studies
There are no directly comparable MRH tools as this is the first developed for application to the transition of care following hospital discharge. Moreover, the PRIME tool calculates risk of MRH rather than solely ADR, as is the case with many existing tools (see online supplementary file for summary of existing tools). The PRIME tool’s discrimination and calibration49 59 are comparable with these existing tools (PRIME C-statistic 0.66 vs C-statistics of other tools, eg, ADRROP60 (0.59), GerontoNet61 (0.70), PADR-EC62 (0.67), BADRI63 (0.73), Trivalle64 (0.70)). The PRIME tool displayed good calibration, while other tools to predict MRH have not reported this important measure, with exception of BADRI (Hosmer-Lemeshow p=0.757, calibration plot not reported).23 63
The PRIME tool provides an absolute patient risk of MRH, whereas other tools indicate a relative patient risk based on scoring systems that round predictor ORs to the nearest integer. While there is merit in making a tool as simple as possible for routine clinical use, it can lead to imprecision of calculating patient risk. Given the widespread use of electronic patient records in the hospital setting, there is no longer a need to compromise the precision of risk calculation to facilitate manual calculations.
The decision curve analysis shows that the PRIME tool offers a higher net benefit compared with existing strategies to target individuals at increased risk of MRH. For instance, the MedSafer17 randomised controlled deprescribing trial includes older patients using more than four regular medicines to determine if a deprescribing intervention reduces adverse drug events 30 days postdischarge. The PRIME tool could offer an improved approach to targeting higher risk individuals for such interventions.
Predictors in the model
The PRIME tool includes a combination of demographic (age, female gender), medication-specific (number of medicines, antiplatelet drug, antidiabetic drug, past ADR), biochemical (sodium level) and social predictors (living alone). This comprehensive set of patient information reflects the complexity of predicting MRH and healthcare use.65 We anticipated the tool to hold a high degree of face validity among both clinicians and patients given that the selection of candidate variables was informed by both. Most of the variables in the tool have been previously demonstrated to be associated with MRH and/or unplanned healthcare utilisation in older adults.2 61 63 66–70 Few studies have investigated living alone as a risk factor for MRH.69 Our study substantiates self-reported evidence of adverse outcomes in older adults living alone following hospital discharge.71 The informal support from co-habiting family could facilitate medicine adherence and promote early recognition and management of MRH.
Our study extends the pool of evidence for poor outcomes associated with low sodium levels, including the postdischarge setting.66 72–74 Even mild hyponatraemia is associated with attention deficits, postural instability and falls.73 This might contribute to patients being less receptive to medicines information around discharge, and less able to tolerate common ADRs such as dizziness without serious consequences.
Over time, patterns of prescribing will change and ongoing updates and validation of the tool is recommended,75 as has been the case for other tools.22 This might include investigating the impact of adding new prognostic factors, such as frailty76 and genetic variants.77
Implications for practitioners and policy makers
The PRIME tool is unique in providing practitioners with a patient’s absolute risk of MRH following hospital discharge, and policy makers with a tool that identifies those that will consume healthcare resources as a consequence. While the tool has been developed to support a safer hospital discharge by better safeguarding patients at high risk of MRH, the variables in the PRIME tool are routinely available and its usefulness could be evaluated in primary care settings.
The absence thus far of an evidence-based mechanism to risk stratify patients could have contributed to the limited effectiveness of previously tested interventions to reduce MRH, in spite of strong face validity.17 78–81 The PRIME tool could improve the efficiency and effectiveness of interventions to reduce MRH postdischarge.82 The integration of the tool into electronic discharge systems could automate the risk calculation and be made available to GPs on discharge summaries. Furthermore, the visibility of patient MRH risk to doctors compiling discharge summaries may stimulate wider positive changes in prescribing practices. For instance, it might prompt an additional check of the accuracy of the prescribed discharge medications given that a high proportion of discharge summaries convey inaccurate information of prescribed medicines. The availability of MRH risk information to GPs could stimulate stronger consideration of whether new patient symptoms are due to disease or MRH. With the advent of Primary Care Networks in the NHS, through which a major focus is prescribing and medicines safety, there is an opportunity to use the PRIME tool to prioritise the action of 7500 new clinical pharmacists being introduced into general practice under NHS England’s £100 million medicines optimisation programme.83
The use of probabilities is an important part of conveying health risk to patients in a comprehensible manner.84 85 Patients would co-own data on their risk of MRH at discharge, which might enhance their awareness that symptoms occurring in the initial weeks postdischarge might be due to MRH rather than the symptom of a new disease. Furthermore, the PRIME tool should aid better informed discussions between patients and doctors with regard to reducing polypharmacy, thus breaking down some barriers to deprescribing.86 Clinicians and patients tend to overestimate the benefits of treatments and underestimate the harms.87 88 Patients having individualised risk information could help to rebalance this skewed risk perception.
Limitations
There are several limitations of this work to be considered. Research pharmacists identifying MRH were not blinded to the study aim, potentially introducing bias. In cases of MRH leading to hospital re-admission, consultants independent of the study assigned causality to the event alongside the research pharmacist, thereby reducing the impact of potential bias.
The PRIME tool was developed excluding patients with ‘possible’ and ‘probable’ MRH in order to strengthen the reliability of the model to detect true MRH cases in real-world clinical practice. This hypothesis needs to be tested in an external validation phase. Although bootstrap is an optimal method for internally validating a prognostic tool, it is not a replacement for the necessary external validation phase prior to implementation. The usability of the tool, once implemented into an electronic system, will also require investigation. Nonetheless, the large cohort investigated in this study using a multicentre, prospective design, in conjunction with a robust approach to MRH verification, and adjustment for model optimism, lends the PRIME tool to favourable re-testing.
Although the PRIME tool’s C-statistic is not dissimilar to other tools for predicting complex outcomes, for example, stroke,89 certain cancers,90 and hospital readmission,91 92 the implications of false positives and false negatives should be considered. While a false positive may result in increased medication scrutiny, the information on risk has potential to cause unnecessary concern to patients and clinicians. Additionally, any intervention will use clinical resource that might have been better placed elsewhere in the healthcare system. False negatives meanwhile limit the tool’s clinical utility to reduce MRH and avoidable healthcare use.
Conclusions
The PRIME tool identifies older patients at high risk of MRH requiring healthcare use following hospital discharge. This could support efforts to target medicines optimisation to reduce MRH and healthcare use. The tool should now be evaluated in new settings.
References
Footnotes
Contributors CR, RS, JH, GD, JMS, KA and TvdC conceived of the PRIME study. KA, JH, GD and CR were co-applicants for the study grant. NP, CR, GD, JMS and KA designed this study. Data analysis was performed by NP, KA, GD and CR and statistical expertise was provided by WB and SN. All authors were involved in preparing the manuscript. All authors reviewed and approved the final manuscript.
Funding This research was funded by the National Institute for Health Research (PB-PG-0711-25094).
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval This research was approved by the National Research Ethics Service, East of England (REC Reference 13/EE/0075).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.