You are here

  • Home
  • Archive
  • Volume 29, Issue 9
  • Validation of automated sepsis surveillance based on the Sepsis-3 clinical criteria against physician record review in a general hospital population: observational study using electronic health records data

Article Text

Article menu
Original research
Validation of automated sepsis surveillance based on the Sepsis-3 clinical criteria against physician record review in a general hospital population: observational study using electronic health records data
  1. John Karlsson Valik1,2,
  2. Logan Ward3,4,
  3. Hideyuki Tanushi2,
  4. Kajsa Müllersdorf1,2,
  5. Anders Ternhag1,2,
  6. Ewa Aufwerber2,
  7. Anna Färnert1,2,
  8. Anders F Johansson5,
  9. Mads Lause Mogensen3,
  10. Brian Pickering6,
  11. Hercules Dalianis7,
  12. Aron Henriksson7,
  13. Vitaly Herasevich6,
  14. Pontus Nauclér1,2
  1. 1 Division of Infectious Diseases, Department of Medicine, Solna (MedS), Karolinska Institutet, Stockholm, Sweden
  2. 2 Department of Infectious Diseases, Karolinska University Hospital, Stockholm, Sweden
  3. 3 Treat Systems ApS, Aalborg, Denmark
  4. 4 Center for Model-based Medical Decision Support, Department of Health Science and Technology, Aalborg University, Aalborg, Denmark
  5. 5 Department of Clinical microbiology and the Laboratory for Molecular Infection Medicine (MIMS), Umeå University, Umeå, Sweden
  6. 6 Department of Anesthesiology and Perioperative medicine, Mayo Clinic, Rochester, Minnesota, USA
  7. 7 Department of Computer and Systems Sciences, Stockholm University, Kista, Sweden
  1. Correspondence to Dr John Karlsson Valik, Division of Infectious Diseases, Department of Medicine, Solna (MedS), Karolinska Institutet, Stockholm, 171 77 Solna, Sweden; john.karlsson.valik{at}ki.se

Abstract

Background Surveillance of sepsis incidence is important for directing resources and evaluating quality-of-care interventions. The aim was to develop and validate a fully-automated Sepsis-3 based surveillance system in non-intensive care wards using electronic health record (EHR) data, and demonstrate utility by determining the burden of hospital-onset sepsis and variations between wards.

Methods A rule-based algorithm was developed using EHR data from a cohort of all adult patients admitted at an academic centre between July 2012 and December 2013. Time in intensive care units was censored. To validate algorithm performance, a stratified random sample of 1000 hospital admissions (674 with and 326 without suspected infection) was classified according to the Sepsis-3 clinical criteria (suspected infection defined as having any culture taken and at least two doses of antimicrobials administered, and an increase in Sequential Organ Failure Assessment (SOFA) score by >2 points) and the likelihood of infection by physician medical record review.

Results In total 82 653 hospital admissions were included. The Sepsis-3 clinical criteria determined by physician review were met in 343 of 1000 episodes. Among them, 313 (91%) had possible, probable or definite infection. Based on this reference, the algorithm achieved sensitivity 0.887 (95% CI: 0.799 to 0.964), specificity 0.985 (95% CI: 0.978 to 0.991), positive predictive value 0.881 (95% CI: 0.833 to 0.926) and negative predictive value 0.986 (95% CI: 0.973 to 0.996). When applied to the total cohort taking into account the sampling proportions of those with and without suspected infection, the algorithm identified 8599 (10.4%) sepsis episodes. The burden of hospital-onset sepsis (>48 hour after admission) and related in-hospital mortality varied between wards.

Conclusions A fully-automated Sepsis-3 based surveillance algorithm using EHR data performed well compared with physician medical record review in non-intensive care wards, and exposed variations in hospital-onset sepsis incidence between wards.

  • adverse events, epidemiology and detection
  • critical care
  • nosocomial infections
  • information technology
  • continuous quality improvement
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjqs-2019-010123

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors Concept and design: PN, JKV, LW, VH, BP, AF, AT, AFJ, MLM, HD, AH. Acquisition, analysis, or interpretation of data: JKV, LW, PN, EA, HT, KM, MLM, HD, AH. Drafting of the manuscript: JKV, PN, LW. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: LW, JKV, PN. Obtained funding: PN, AF, AFJ, MLM, HD, JKV, BP, VH.

    • Funding The work was supported by Vinnova (grant 2016-00563). JKV was supported by Region Stockholm (combined clinical residency and PhD training program). PN was supported by Region Stockholm (clinical research appointment). JKV, PN, VH and BP received the Mayo Clinic-Karolinska Institutet Collaborative Travel Award 2017.

    • Competing interests LW and MLM are employees of Treat Systems ApS (Aalborg, Denmark). Treat Systems produces medical decision support systems for antimicrobial and microbiological diagnostic stewardship.

    • Patient consent for publication Not required.

    • Ethics approval The study was approved by the Regional Ethical Review Board in Stockholm under permission no. 2016/2309-32 and 2012/1838-31/3

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement Data from deidentified electronic health records are not freely available due to protection of the personal integrity of the participants. Access to patient level data requires a Swedish ethical permit and an agreement with the research organisation, Department of Computer and Systems Sciences, Stockholm University, holder of the data.

    Linked Articles