Article Text

Download PDFPDF
Validation of new ICD-10-based patient safety indicators for identification of in-hospital complications in surgical patients: a study of diagnostic accuracy
  1. Daniel I McIsaac1,2,3,
  2. Gavin M Hamilton1,
  3. Karim Abdulla1,
  4. Luke T Lavallée2,4,
  5. Husien Moloo2,5,
  6. Chris Pysyk1,
  7. Jocelyn Tufts6,
  8. William A Ghali7,8,
  9. Alan J Forster2,3
  1. 1Departments of Anesthesiology and Pain Medicine, University of Ottawa, The Ottawa Hospital, Ottawa, Ontario, Canada
  2. 2Clinical Epidemiology, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
  3. 3School of Epidemiology & Public Health, University of Ottawa, Ottawa, Ontario, Canada
  4. 4Department of Surgery, Division of Urology, University of Ottawa, Ottawa, Ontario, Canada
  5. 5Department of Surgery, Division of General Surgery, University of Ottawa, Ottawa, Ontario, Canada
  6. 6Performance Measurement, The Ottawa Hospital, Ottawa, Ontario, Canada
  7. 7Department of Community Health Sciences, Calgary Institute for Population and Public Health, University of Calgary, Calgary, Alberta, Canada
  8. 8Department of Medicine, Calgary Institute for Population and Public Health, University of Calgary, Calgary, Alberta, Canada
  1. Correspondence to Dr Daniel I McIsaac, Anesthesiology & Pain Medicine, University of Ottawa, 1050 Carling Ave, Room B311, Ottawa, ON K1Y 4E9, Canada; dmcisaac{at}


Objective Administrative data systems are used to identify hospital-based patient safety events; few studies evaluate their accuracy. We assessed the accuracy of a new set of patient safety indicators (PSIs; designed to identify in hospital complications).

Study design Prospectively defined analysis of registry data (1 April 2010–29 February 2016) in a Canadian hospital network. Assignment of complications was by two methods independently. The National Surgical Quality Improvement Programme (NSQIP) database was the clinical reference standard (primary outcome=any in-hospital NSQIP complication); PSI clusters were assigned using International Classification of Disease (ICD-10) codes in the discharge abstract. Our primary analysis assessed the accuracy of any PSI condition compared with any complication in the NSQIP; secondary analysis evaluated accuracy of complication-specific PSIs.

Patients All inpatient surgical cases captured in NSQIP data.

Analysis We assessed the accuracy of PSIs (with NSQIP as reference standard) using positive and negative predictive values (PPV/NPV), as well as positive and negative likelihood ratios (±LR).

Results We identified 12 898 linked episodes of care. Complications were identified by PSIs and NSQIP in 2415 (18.7%) and 2885 (22.4%) episodes, respectively. The presence of any PSI code had a PPV of 0.55 (95% CI 0.53 to 0.57) and NPV of 0.93 (95% CI 0.92 to 0.93); +LR 6.41 (95% CI 6.01 to 6.84) and −LR 0.40 (95% CI 0.37 to 0.42). Subgroup analyses (by surgery type and urgency) showed similar performance. Complication-specific PSIs had high NPVs (95% CI 0.92 to 0.99), but low to moderate PPVs (0.13–0.61).

Conclusion Validation of the ICD-10 PSI system suggests applicability as a first screening step, integrated with data from other sources, to produce an adverse event detection pathway that informs learning healthcare systems. However, accuracy was insufficient to directly identify or rule out individual-level complications.

  • Adverse events, epidemiology and detection
  • Chart review methodologies
  • Healthcare quality improvement
  • Incident reporting
  • Surgery
View Full Text

Statistics from


  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement No additional data are available.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles