Table 1

Summary of studies comparing automated harm-detection methods with gold standard chart review

Reference	Patient sample and time frame	Sampling strategy^*	Specialty	Events identified	Automated event dataset sample size	Comparison event dataset sample size
Field-defined
Nebeker et al18	Adults 2001 and 2003	Random	Unknown	Adverse drug events	3987 admissions	3987 admissions
Zhan et al17	Medicare benefic. 2002 to 2004	Random	General Surgery	Adverse event: specifically postoperative deep venous thrombosis and/or pulmonary embolism	20 868 hospital discharges identified as surgical patients	20 868 hospital discharges identified as surgical patients
Brossette et al44	Unknown 1–3 Dec 2003 and 26–29 Apr 2004	Sequential	Unknown	Infection	907 admissions	907 admissions
Hougland et al30	Adults 1 Jan 2001 to 31 Dec 2001	Random, Flagged sample (from records with at least one flagged adverse drug event code)	Unknown	Adverse drug events	3103 inpatients: 1961 random, 1142 flagged	Unknown
Polancich et al15	Unknown	Unknown	Unknown	Hospital acquired decubitus ulcers	Unknown	123 charts from patients with PSI-identified decubitus ulcers
Dormann et al26	Adults 1 Sept 2000 to 28 Feb 2001	Sequential	Gastroenterology	Adverse drug events	474 admissions of 377 patients; 109 ADEs	474 admissions of 377 patients; 109 adverse drug events
Trick et al20	Adults 1 Sept 2001 to 28 Feb 2002	Sequential	Unknown	Infection	135 positive blood cultures	144 positive blood cultures
Levy et al23	All age groups 1 Apr 1997 to 31 May 1997	Sequential	General Medical	Adverse drug events	199 admissions (192 patients)	199 admissions
Azaz-Livshits et al22	All age groups 1 Apr 1995 to 31 May 1995	Sequential	General Medical	Adverse drug events	153 admissions	153 admissions
Jha et al32	Adults 1 Oct 1994 to 31 May 1995	Sequential	MICU, SICU, General Medical, General Surgical	Adverse drug events	21 964 patient-days	21 964 patient-days
NLP
Penz et al47	Adults 1 Jun 1999 to 31 Dec 2004	Sequential	MICU, SICU and other (placement of CVC)	Adverse events related to central venous catheter placement	316 patient records	40 patients records (10 very low probability^† records, 30 high probability)
Forster et al34	Adults FY 2002	Random	General Medical, General Surgical	Adverse event	245 patients	245 patients
Melton and Hripcsak48	Unknown 1996–2000	Random (charts), Sequential (electronic discharge summaries)	Unknown	Adverse events: specifically 45 NYPORTS event types	1000 charts, 57 422 electronic discharge summaries	1000 charts
Murff et al33	Adults 1 Jan 2000 to 30 Jun 2000	Random (Cohort 1), Sequential (Cohort 2)	General Medical, Medicine subspecialties	Adverse drug events, adverse events, diagnostic errors, operative complications, falls	Cohort 1: 424 admissions. Cohort 2: 2826 admissions	Cohort 1: 295 Cohort 2: 145 Complex sampling/subsampling and manual review process

↵* Sampling strategy refers to the method by which charts were chosen to be screened by the automated tool. Unless specifically noted, the same sampling strategy also applies to the gold standard method.
↵† A scoring system was developed by the authors to reflect the probability of the adverse event in question relating to the central venous catheter placement. This system is described in the text and in table 1 of the paper.
CVC, central venous catheter; MICU, Medical Intensive Care Unit; NYPORTS, New York Patient Occurrence Reporting and Tracking System; PSI, patient safety indicators; SICU, Surgical Intensive Care Unit.