Article Text
PDF
Abstract
Objective Evaluate predictive performance of an electronic health record (EHR)-based, inpatient 6-month mortality risk model developed to trigger palliative care consultation among patient groups stratified by age, race, ethnicity, insurance and socioeconomic status (SES), which may vary due to social forces (eg, racism) that shape health, healthcare and health data.
Design Retrospective evaluation of prediction model.
Setting Three urban hospitals within a single health system.
Participants All patients ≥18 years admitted between 1 January and 31 December 2017, excluding observation, obstetric, rehabilitation and hospice (n=58 464 encounters, 41 327 patients).
Main outcome measures General performance metrics (c-statistic, integrated calibration index (ICI), Brier Score) and additional measures relevant to health equity (accuracy, false positive rate (FPR), false negative rate (FNR)).
Results For black versus non-Hispanic white patients, the model’s accuracy was higher (0.051, 95% CI 0.044 to 0.059), FPR lower (−0.060, 95% CI −0.067 to −0.052) and FNR higher (0.049, 95% CI 0.023 to 0.078). A similar pattern was observed among patients who were Hispanic, younger, with Medicaid/missing insurance, or living in low SES zip codes. No consistent differences emerged in c-statistic, ICI or Brier Score. Younger age had the second-largest effect size in the mortality prediction model, and there were large standardised group differences in age (eg, 0.32 for non-Hispanic white versus black patients), suggesting age may contribute to systematic differences in the predicted probabilities between groups.
Conclusions An EHR-based mortality risk model was less likely to identify some marginalised patients as potentially benefiting from palliative care, with younger age pinpointed as a possible mechanism. Evaluating predictive performance is a critical preliminary step in addressing algorithmic inequities in healthcare, which must also include evaluating clinical impact, and governance and regulatory structures for oversight, monitoring and accountability.
- evaluation methodology
- decision support, computerized
- information technology
Data availability statement
No data are available.
Statistics from Altmetric.com
Data availability statement
No data are available.
Footnotes
Contributors ST, CC, KL, SH, NE, MD and KC contributed to the conception and design of the study, acquisition of data, interpretation of results, and manuscript drafting and substantive revision. ST and CC had full access to all the data in the study and conducted statistical analyses. All authors approved the final draft. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. ST acts as guarantor.
Funding This study was funded by the US National Library of Medicine (Grant number: F31LM013403).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
© Author(s) (or their employer(s)) 2023. No commercial re-use. See rights and permissions. Published by BMJ.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.