Objective: To assess the feasibility of implementing the National Surgical Quality Improvement Program (NSQIP) methodology in non-VA hospitals.
Summary background data: Using data adjusted for patient preoperative risk, the NSQIP compares the performance of all VA hospitals performing major surgery and anonymously compares these hospitals using the ratio of observed to expected adverse events. These results are provided to each hospital and used to identify areas for improvement. Since the NSQIP's inception in 1994, the VA has reported consistent improvements in all surgery performance measures. Given the success of the NSQIP within the VA, as well as the lack of a comparable system in non-VA hospitals, this pilot study was undertaken to test the applicability of the NSQIP models and methodology in the nonfederal sector.
Methods: Beginning in 1999, three academic medical centers (Emory University, Atlanta, GA; University of Michigan, Ann Arbor, MI; University of Kentucky, Lexington, KY) volunteered the time of a dedicated surgical nurse reviewer who was trained in NSQIP methodology. At each academic center, these nurse reviewers used NSQIP protocols to abstract clinical data from general surgery and vascular surgery patients. Data were manually collected and then transmitted via the Internet to a secure web site developed by the NSQIP. These data were compared to the data for general and vascular surgery patients collected during a concurrent time period (10/99 to 9/00) within the VA by the NSQIP. Logistic regression models were developed for both non-VA and VA hospital data. To assess the models' predictive values, C-indices (0.5 = no prediction; 1.0 = perfect prediction) were calculated after applying the models to the non-VA as well as the VA databases.
Results: Data from 2,747 (general surgery 2,251; vascular surgery 496) non-VA hospital cases were compared to data from 41,360 (general surgery 31,393; vascular surgery 9,967) VA cases. The bivariate relationships between individual risk factors and 30-day mortality or morbidity were similar in the non-VA and VA patient populations for over 66% of the risk variables. C-indices of 0.942 (general surgery), 0.915 (vascular surgery), and 0.934 (general plus vascular surgery) were obtained following application of the VA NSQIP mortality model to the non-VA patient data. Lower C-indices (0.778, general surgery; 0.638, vascular surgery; 0.760, general plus vascular surgery) were obtained following application of the VA NSQIP morbidity model to the non-VA patient data. Although the non-VA sample size was smaller than the VA, preliminary analysis suggested no differences in risk-adjusted mortality between the non-VA and VA cohorts. CONCLUSIONS With some adjustments, the NSQIP methodology can be implemented and generates reasonable predictive models within non-VA hospitals.