Does it matter what a hospital is “high volume” for? Specificity of hospital volume-outcome associations for surgical procedures: analysis of administrative data*
- 1Department of Surgery, University of Toronto, 200 Elizabeth Street, 9EN-236A, Toronto, ON M5G 2C4, Canada
- 2Department of Surgery, University of Minnesota Cancer Center, MMC 806, 420, Delaware Street SE, Minneapolis, MN 55455, USA
- Correspondence to: D R Urbach Department of Surgery, University of Toronto, 200 Elizabeth Street, 9EN-236A, Toronto, ON M5G 2C4, Canada;
- Accepted 31 December 2004
Objective: To determine whether the improved outcome of a surgical procedure in high volume hospitals is specific to the volume of the same procedure.
Design and setting: Analysis of secondary data in Ontario, Canada.
Participants: Patients having an oesophagectomy, colorectal resection for cancer, pancreaticoduodenectomy, major lung resection for cancer, or repair of an unruptured abdominal aortic aneurysm between 1994 and 1999.
Main outcome measures: Odds ratio for death within 30 days of surgery in relation to the hospital volume of the same surgical procedure and the hospital volume of the other four procedures. Estimates were adjusted for age, sex, and comorbidity and accounted for hospital level clustering.
Results: With the exception of colorectal resection, 30 day mortality seemed to be inversely related not only to the hospital volume of the same procedure but also to the hospital volume of most of the other procedures. In some cases the effect of the volume of a different procedure was stronger than the effect of the volume of the same procedure. For example, the association of mortality from pancreaticoduodenectomy with hospital volume of lung resection (odds ratio for death in hospitals with a high volume of lung resection compared with low volume 0.36, 95% confidence interval 0.23 to 0.57) was much stronger than the association of mortality from pancreaticoduodenectomy with hospital volume of pancreaticoduodenectomy (0.76, 0.44 to 1.32).
Conclusion: The inverse association between high volume of procedure and risk of operative death is not specific to the volume of the procedure being studied.
Evidence that the short term outcomes of complex surgical procedures are better in hospitals that do high volumes of such procedures has prompted some authors to suggest that high risk surgery should be regionalised at high volume hospitals.1–5 Relatively little research has been done into the underlying mechanisms and the potential role of selection bias in surgical volume-outcome associations.6–10 Health policy measures advocating volume based regionalisation are, for the most part, predicated on the overwhelming empirical evidence of hospital volume-outcome associations.11 In general, policy initiatives have proposed that patients needing certain high risk surgical procedures should have them done in a hospital that performs a large volume of similar procedures.
The findings of volume-outcome studies are usually interpreted in the light of the conceptual framework of quality in health care proposed by Donabedian: structures, processes, and outcomes.12 High volume hospitals are assumed to have structural characteristics associated with better quality of care, and providers in these hospitals are thought to improve their processes of care through experience in providing complex care. Central to this framework is an implied linkage between the volume of a specific surgical procedure done in a hospital and the outcome of the same surgical procedure. The finding of improved outcome after pancreaticoduodenectomy in high volume hospitals has been uniformly attributed to the high volume of pancreaticoduodenectomy,3,13–15 not the volume of a different complex procedure, the volume of all complex procedures, or other hospital characteristics. Whether the volume-outcome association is unique to the combination of the volume and the outcome of the same procedure has never been tested. We sought to answer the question of whether the improved outcome observed in high volume hospitals was unique to the volume of the procedure of which the outcome is being assessed.
Sources of data
We used abstracted electronic records to identify hospital separations in Ontario, Canada, between 1 April 1994 and 31 March 1999 and linked these records to a database of vital statistics to ascertain the vital status of the individual patients by using an anonymous unique identifier. The reliability of coding surgical procedures in the Ontario health databases is good, with 88–96% agreement between databases for procedures such as cholecystectomy and hysterectomy.16
We examined the outcome of five surgical procedures in relation to volume: oesophagectomy, excision of a segment of the colon or rectum for colorectal cancer, pancreaticoduodenectomy, major lung resection (lobectomy or pneumonectomy) for lung cancer, and repair of an unruptured abdominal aortic aneurysm (AAA). We specifically chose these procedures because they are complex, are associated with an appreciable risk of operative death, and have been identified as potential targets for volume based regionalisation. The codes used for identifying these procedures have been described elsewhere.17
Measurement of hospital volume and outcome
Where the code used to identify a hospital changed during the study period owing to corporate restructuring or amalgamation, we identified the hospital by using the institution code in effect at the end of the study period. We calculated the average hospital volume of each procedure on the basis of the number of identical procedures done at the hospital over the five year study period. We dichotomised hospitals into two volume categories (high volume hospitals and low volume hospitals) at the median average annual hospital volume, such that patients having a surgical procedure were divided into two equal groups. As the distribution of average annual hospital volumes for other procedures usually differed from the distribution of hospital volumes of the procedure whose outcome was being studied, cut-off points used to separate high volume and low volume hospitals varied according to the procedure volume specified as an exposure variable. The outcome measure for all analyses was death within 30 days after the surgical procedure, regardless of place or location.
The overall strategy of the analyses was to model the association of hospital procedure volume with 30 day mortality, adjusting for the patient level characteristics of age, sex, and comorbidity. Age was represented as a continuous variable in the analyses, as was comorbidity (using a modified Charlson comorbidity score).18,19 For each surgical procedure (for example, oesophagectomy), we first assessed the outcome (death within 30 days of surgery) in relation to whether a patient had surgery at a hospital that did a high or a low volume of the same procedure (for example, oesophagectomy). Next, we assessed the outcome among those patients having a procedure (for example, oesophagectomy) according to whether they had surgery at a hospital that did a high or a low volume of each of the other four procedures (for example, colorectal resection, pancreaticoduodenectomy, lung resection, and AAA repair). In total, we created five separate cohorts of patients (one for each surgical procedure) and did five volume-outcome analyses for each of the five procedure based cohorts (one analysis for each method of determining categories of hospital procedure volume). Analyses specifying volume as a continuous variable or as quarters yielded similar results and are not reported here.
We used generalised linear models for binary outcomes to estimate associations between hospital volume and 30 day mortality. To account for the effects of clustering at the hospital level, we estimated the model parameters and standard errors by using generalised estimating equations,20,21 with the hospital identifier specified as the cluster level variable in the adjusted analyses, which also controlled for age, sex, and comorbidity. To assess whether hospital procedure volumes were correlated, we estimated Spearman rank correlation coefficients for hospital procedure volume according to quarters of volume. We used SAS version 8.2 software for UNIX for all analyses.
Patients and hospitals
During the five year study period, 31 632 patients had one of the five surgical procedures of interest (table 1). Patients undergoing oesophagectomy seemed to have the highest burden of comorbid illness (median Charlson score 4), whereas those having colorectal cancer resection or repair of an unruptured AAA had fewer comorbid conditions (median Charlson score 0). The largest preponderance of male patients was among those having an oesophagectomy (73.4%) or repair of unruptured AAA (82.3%). Mortality within 30 days of surgery ranged from 3.8% (excision of colon or rectum for cancer) to 13.4% (oesophagectomy).
Table 2 shows associations between volume and outcome for the five surgical procedures. In this table, the rows indicate the procedure of which the outcome is being assessed, and the columns indicate the procedure that was used to define hospital volume. For example, the first column of data in the first row represents the outcome of oesophagectomy according to the hospital volume of oesophagectomy. The second column of data in the first row represents the outcome of oesophagectomy according to the hospital volume of colorectal resection. Comparisons of operative mortality by hospital volume for the same procedure are indicated in bold along the diagonal.
Association of outcome of procedure with volume of same procedure
Hospital volume and 30 day mortality were significantly associated for lung resection (adjusted odds ratio for death at high volume hospitals compared with low volume hospitals 0.64, 95% confidence interval 0.44 to 0.94) and AAA repair (0.62, 0.46 to 0.83). Although the point estimates of the association of volume and outcome for oesophagectomy (0.60, 0.30 to 1.20) and pancreaticoduodenectomy (0.76, 0.44 to 1.32) were consistent with an inverse relation between volume and outcome, the number of patients who had these procedures was relatively small and the confidence intervals included values consistent with no association. We found little evidence of an association between volume and outcome for colorectal resection (0.98, 0.83 to 1.16).
Association of outcome of procedure with volume of different procedure
We also examined the effect on operative mortality of the hospital volume of procedures other than the one for which the outcome was being measured. These comparisons are indicated by the non-bold data off the diagonal in table 2. In many instances, 30 day mortality was associated with the hospital volume of different procedures. This is illustrated by the fact that many of the off-diagonal odds ratio estimates are less than 1.0, indicating improved outcome in high volume hospitals regardless of the procedure for which a hospital was “high volume.” For example, the reduction in 30 day mortality after pancreaticoduodenectomy in hospitals that were high volume hospitals for AAA repair (odds ratio 0.75, 0.45 to 1.27) was similar to the reduction in 30 day mortality after pancreaticoduodenectomy in hospitals that were high volume hospitals for pancreaticoduodenectomy (0.76, 0.44 to 1.32).
The association with the volume of a different procedure was occasionally stronger than with that of the same procedure. For example, the reduction in 30 day mortality after pancreaticoduodenectomy in hospitals that were high volume hospitals for lung resection (0.36, 0.23 to 0.57) was much stronger than the reduction in 30 day mortality after pancreaticoduodenectomy in hospitals that were high volume hospitals for pancreaticoduodenectomy (0.76, 0.44 to 1.32; table 2).
Correlation of hospital procedure volumes
The correlation coefficients for hospital volume for the five procedures we studied ranged from 0.17 (oesophagectomy and colorectal resection) to 0.73 (oesophagectomy and lung resection).
Because volume-outcome studies assess the outcome of a surgical procedure in relation to the volume of the same surgical procedure, they imply a measure of specificity of volume-outcome associations. The consistent finding of associations between the volume of a procedure and surgical outcome has been interpreted as empirical evidence that the outcomes of surgical procedures are better in hospitals with higher volumes of similar procedures. Under this interpretation, regionalisation of patients needing complex surgical procedures to hospitals that do a high volume of those procedures would be expected to improve patient outcomes regardless of the underlying causal mechanism of the volume-outcome association.
We found that the short term outcomes of some complex surgical procedures were better in hospitals with a higher volume of the same procedure. However, we found that in many cases outcomes were also better in hospitals with high volumes of different procedures. Several possible explanations for this finding exist. The volumes of some surgical procedures done within a hospital are correlated. For example, it is not surprising that the outcomes of oesophagectomies and pulmonary resections are correlated with the hospital volume of the other procedure, as the hospital volume of oesophagectomy was highly correlated with the volume of pulmonary resection. General thoracic surgeons do both of these procedures and may be clustered in specific hospitals. Alternatively, the lack of specificity of volume-outcome associations may indicate a more general relation between the overall volume of complex surgery done in a hospital and outcomes. A hospital that does a high volume of any complex procedure is likely to have certain characteristics, such as location in a metropolitan area, status as a teaching hospital, and availability of specialised resources such as intensive care units staffed by full time specialists in intensive care, on-site coronary revascularisation facilities, and interventional radiology.
Limitations of the study
Our finding that volume-outcome associations for hospital procedures are not specific to unique combinations of the volume and outcome of the same procedure cannot plausibly be explained by problems with data quality, unmeasured severity of illness, or other well described limitations of secondary data analysis.22,23 Can our results be explained by confounding, in that if a hospital is high volume for one procedure it is likely to be high volume for another? Although we observed modest correlations between procedure volumes within hospitals, we do not believe that our results can be explained entirely by confounding. For example, although the hospital procedure volumes of oesophagectomy and colorectal resection were only weakly correlated, the volume-outcome associations for oesophagectomy were virtually identical regardless of whether we used oesophagectomy or colorectal resection to define hospital volume.
Our methods were similar to those of most volume-outcome studies. Our use of 30 day mortality as an outcome measure is consistent with a large body of volume-outcome literature that focuses on short term outcome, rather than other outcomes such as cure of cancer, which may be more sensitive measures of the quality of care. Although limitations such as incomplete data on comorbid conditions or misclassification of hospital volume may have affected the validity of our estimates of volume-outcome associations, we have no reason to suspect that they would cause spurious associations between the volume of one procedure and the outcome of another. Most sources of error are non-differential with respect to exposure and outcome and for a binary outcome would be expected to bias estimates of association towards the null hypothesis of no association.24
Implications for health policy
If the improved outcome in high volume hospitals is not specifically related to the volume of the same procedure, but is related to the shared structure and process characteristics of the large hospitals that typically do a high volume of complex surgical procedures, what do our findings say about volume based regionalisation policies? Two possible approaches exist. One is to accept the lack of specificity or understanding of the mechanisms and continue to pursue volume based regionalisation in the light of the strong empirical evidence of volume-outcome associations for many complex procedures.
Another approach is to revisit the conceptual framework underlying volume based regionalisation. Volume-outcome associations for complex surgical procedures may be less a reflection of extraordinarily good care in high volume hospitals than an indication of deficient care in poorly supported small and rural hospitals. If so, regionalisation at large hospitals may benefit the relatively small segment of the population needing complex elective surgery but would accomplish little for the many patients admitted to small and rural hospitals for emergency conditions or medical diagnoses, especially if regionalisation leads to further erosion of resources for managing complex medical problems at smaller hospitals.
Volume-outcome associations for hospital procedures are not specific to the volume and the outcome of the same procedure. Our data do not support health policy measures predicated on referring patients having a certain surgical procedure to hospitals that do a high volume of the same procedure. A more rational strategy might be simply to regionalise all complex operations at large hospitals. Alternatively, increased allocation of resources to smaller hospitals and targeted quality improvement programmes might reduce some of the variation in short term surgical outcomes across hospitals.
We thank Minh Le Duong-Hua for cleaning and linking the datasets. This research was presented as a poster at the 2002 Annual Conference of the Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario, Canada, 28 September 2002.
CONTRIBUTORS DRU and NNB had the original idea and designed and conducted the study. DRU wrote the first draft of the manuscript, which was edited by NNB. DRU is the guarantor.
Funding: This research was supported by an internal investigator initiated grant from the Institute for Clinical Evaluative Sciences and by the physicians of Ontario through the Physicians’ Services Incorporated Foundation. DRU is a career scientist of the Ontario Ministry of Health and Long-Term Care, Health Research Personnel Development Program.
Competing interests: None declared. Ethical approval: This research was conducted with the approval of the research ethics board of Sunnybrook and Women’s College Health Sciences Centre.
↵* This is a reprint of a paper that appeared in the