Article Text

Download PDFPDF

Anastomotic leakage as an outcome measure for quality of colorectal cancer surgery
  1. H S Snijders1,
  2. D Henneman1,
  3. N L van Leersum1,
  4. M ten Berge1,
  5. M Fiocco1,
  6. T M Karsten2,
  7. K Havenga3,
  8. T Wiggers3,
  9. J W Dekker4,
  10. R A E M Tollenaar1,
  11. M W J M Wouters5
  1. 1Department of Surgery, Leiden University Medical Centre, Leiden, The Netherlands
  2. 2Department of Surgery, Onze Lieve Vrouwen Gasthuis, Amsterdam, The Netherlands
  3. 3Department of Surgery, Academic Medical Centre, Groningen, The Netherlands
  4. 4Department of Surgery, Reinier de Graaf Hospital, Delft, The Netherlands
  5. 5Department of Surgery, Netherlands Cancer Institute-Antoni van Leeuwenhoek Hospital, Amsterdam, The Netherlands
  1. Correspondence to H S Snijders, Department of Surgery, Leiden University Medical Centre, PO Box 9600, Leiden 2300 RC, The Netherlands; h.s.snijders{at}lumc.nl

Abstract

Introduction When comparing mortality rates between hospitals to explore hospital performance, there is an important role for adjustment for differences in case-mix. Identifying outcome measures that are less influenced by differences in case-mix may be valuable. The main goal of this study was to explore whether hospital differences in anastomotic leakage (AL) and postoperative mortality are due to differences in case-mix or to differences in treatment factors.

Methods Data of the Dutch Surgical Colorectal Audit were used. Case-mix factors and treatment-related factors were identified from the literature and their association with AL and mortality were analysed with logistic regression. Hospital differences in observed AL and mortality rates, and adjusted rates based on the logistic regression models were shown. The reduction in hospital variance after adjustment was analysed with Levene's test for equality of variances.

Results 17 of 22 case-mix factors and 4 of 11 treatment factors related to AL derived from the literature were available in the database. Variation in observed AL rates between hospitals was large with a maximum rate of 17%. This variation could not be attributed to differences in case-mix but more to differences in treatment factors. Hospital variation in observed mortality rates was significantly reduced after adjustment for differences in case-mix.

Conclusions Hospital variation in AL is relatively independent of differences in case-mix. In contrast to ‘postoperative mortality’ the observed AL rates of hospitals evaluated in our study were only slightly affected after adjustment for case-mix factors. Therefore, AL rates may be suitable as an outcome indicator for measurement of surgical quality of care.

  • Audit and feedback
  • Mortality (standardized mortality ratios)
  • Surgery
  • Quality measurement

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Nowadays there is a growing public interest in quality of medical and surgical care, with an increasing need for outcome measures that represent hospital performance. The outcome measure postoperative mortality is often used to benchmark surgical performance.1–3 When comparing mortality rates between hospitals, there is an important role for risk adjustment.4 ,5 Observed variations in mortality may be caused by differences in patient and tumour characteristics (case-mix), and high-risk patients may not be evenly distributed between hospitals.6

However, valid case-mix adjustments require a substantial amount of reliable data collected on a patient level. These data are rarely available and require a substantial registration effort. Therefore, it may be valuable to identify outcome measures that are less influenced by differences in case-mix and represent the actual differences in quality of care processes.

Colorectal cancer is a significant source of mortality, with nearly 10 000 new cases diagnosed in the Netherlands each year.7 The cornerstone of this treatment is surgical resection. Patients undergoing surgical resection have a considerable risk for postoperative complications, which can lead to significant morbidity, mortality and large costs. Internationally, several quality improvement programmes have been initiated to reduce postoperative complications after colorectal surgery.

Anastomotic leakage (AL) is one of the most feared complications after colorectal surgery, often causing prolonged hospital stay, morbidity, mortality and possibly worse oncological outcomes.8 The percentage of patients developing AL depends on multiple factors. In the literature, several elements have been identified as risk factors. These can be patient or tumour related, often referred to as case-mix, such as height of the anastomosis, a malnourished status, steroid use and male gender.9–13 Treatment-related factors such as surgeons’ experience, operative duration, blood loss, preoperative radiation and a defunctioning stoma have also been shown to be associated with the occurrence of AL.9–13

The aim of this study was to explore whether hospital differences in AL rates are related to differences in case-mix. We compared the role of case-mix adjustment for AL and postoperative mortality. With this objective, we drew up the following research questions:

  1. Which case-mix and treatment-related risk factors are associated with AL and postoperative mortality after colorectal surgery?

  2. What are the differences in AL and mortality rates between hospitals and are these due to differences in case-mix or due to differences in treatment patterns?

Methods

Patients

Data were derived from the Dutch Surgical Colorectal Audit (DSCA), a national quality improvement project in which over 200 variables concerning the patient, comorbidity, diagnostics, disease-specific details, treatment and outcomes are collected prospectively. The DSCA contains data of patients registered by 92 hospitals (all hospitals performing colorectal cancer surgery). The dataset is disease specific for colorectal cancer and shows a nearly 100% accordance on most items, including AL on validation against the National Cancer Registry dataset.14

All patients undergoing resection for primary colorectal cancer between 1 January 2009 and 31 December 2011 and registered in the DSCA before 15 March 2012 were evaluated. Minimal data requirements for inclusion in analyses were information on tumour location, date of surgery and mortality. Patients with metastases at the time of primary surgery and resections for multiple synchronous colorectal tumours were excluded because these represent subgroups of patients with other treatment perspectives and subsequent different expected outcomes. Also, patients in whom a primary end colostomy was constructed were excluded from the analysis.

Risk factors

Since part of the dataset of the DSCA was designed with the objective of performing case-mix adjustment, particularly for postoperative mortality, variables were determined as risk factors for postoperative mortality at an early stage of development of the dataset. These factors were based on existing evidence on potential risk factors for mortality and determined by an expert panel using a Delphi method.6

To assess whether there are additional case-mix and treatment-related risk factors that need to be taken into account when adjusting for AL, we performed a systematic search of the literature published between 1990 and 2012 on biomedical bibliographical databases PubMed and the Cochrane Library. The search headings ‘anastomotic leak and colorectal surgery’ were used in combination with the keyword ‘risk factor’. The ‘related articles’ function was used to expand the search. References from the articles were also used when appropriate. Letters, reviews without original data, non-English language papers, overlapping patient populations and animal studies were excluded.

From the articles retrieved from the literature search, different risk factors for AL were selected. A distinction was made between patient and tumour related factors (case-mix factors) and treatment-related factors. We selected risk factors with a statistical significance of p<0.05, which were analysed with multivariate logistic regression.

Outcomes

Various definitions of AL have been previously presented.15 The definition of AL in this study was ‘a clinically relevant anastomotic leak requiring a reintervention’. Both radiological and surgical reinterventions were included. Postoperative mortality was defined as ‘death during postoperative hospital stay or within 30 days after the date of surgery’.

Analyses

The association of case-mix and treatment factors, and AL and mortality were tested with multivariate logistic regression models. Separate models were used for each outcome.

To analyse the differences in AL and mortality between hospitals and investigate whether these were due to differences in case-mix or due to differences in treatment patterns we applied three different models. model 1: unadjusted (observed) variation in outcome; model 2: adjusted for patient (case-mix) characteristics; model 3: ‘adjusted’ for case-mix and treatment characteristics. Adjustment was performed by calculating expected outcomes (E) using case-mix (model 2), and case-mix and treatment (model 3) coefficients from the regression analysis. Next, for each hospital, the observed percentage (O) was divided by the expected value (E) and multiplied by the overall mean (observed/E* mean) to obtain the adjusted percentages.

Hospital differences in AL and mortality rates before and after adjustment were plotted in a graph; a summary measure of the between-hospital variance was given with ranges and SDs. The reduction in between-centre variance after adjustment for (model 2) case-mix and (model 3) case-mix and treatment factors was analysed with Levene's test for equality of variances. A p<0.05 was considered statistically significant.

Furthermore, a mixed logistic regression model with hospitals as random effects was performed. A likelihood ratio test was used to test whether the variance of the random effects was statistically significant after adjustment for case-mix and treatment factors.

Hospitals with more than 15% missing case-mix factors were excluded from multivariate analyses. All statistical analyses were performed in PASW Statistics, Rel. 18.0.2009 and R V.2.14.16

Results

On 15 March 2012, 92 hospitals (8 university, 47 teaching and 37 non-teaching hospitals) registered a total of 25 555 eligible patients with primary colorectal cancer with a date of surgery between 1 January 2009 and 31 December 2011 in the DSCA. Nine hospitals had more than 15% missing case-mix factors in total and were therefore excluded (n=1460). After additional exclusion of patients with multiple synchronous tumours (n=598), distant metastases (n=2032) and without an anastomosis (n=5480), a total of 15 236 patients were included in the analysis. Characteristics of the included patients are shown in table 1.

Table 1

Risk factors for AL described in the literature and available patient and treatment characteristics of included patients in the DSCA

Of all patients, 1207 (8%) developed AL and 525 (3.4%) died within 30 days or during hospital admission.

Risk factors

The literature search gave a total of 39 studies describing risk factors for AL.8 ,10–13 17–49 In total, 22 case-mix factors and 11 treatment-related factors were identified. Table 1 shows the results.

The case-mix factors described most frequently were gender, American Society of Anesthesiologists (ASA) score and location of the tumour and/or anastomosis. Treatment factors often described were blood loss/transfusion, duration of the operation and the use of a defunctioning stoma.

Of the 22 case-mix factors for AL identified in literature, 17 were available in the DSCA. The database had no information on the factors weight loss, nutrition status, alcohol abuse, smoking and leukocytosis. Treatment factors were less often available; 4 out of 11 were available in the dataset.

The case-mix and treatment-related risk factors that were found for AL in the literature were similar to those that were used for risk adjustment for postoperative mortality in the DSCA dataset.

A multivariate analysis was performed to investigate the association of case-mix and treatment factors with AL and postoperative mortality. The results of the analysis are shown in table 2.

Table 2

Case-mix and treatment factors included in the multivariate logistic regression model for AL and mortality after colon and rectal carcinoma resections

Individual case-mix factors predicting AL were male gender, urgency of the resection, renal disease and tumour location. Treatment-related factors associated with AL were short preoperative radiotherapy, the absence of a defunctioning stoma and postoperative blood transfusion. For postoperative mortality the case-mix factors age, gender, ASA score, pulmonary disease, tumour location sigmoid and urgency of the resection were individual predicting factors. Treatment-related factors were chemo-radiotherapy and blood transfusion.

Hospital variation

Anastomotic leakage

Unadjusted hospital variation in AL rates was large: the hospital with the lowest percentage had an AL rate of 0% (n=0/166); the hospital with the highest percentage had an AL rate of 18% (n=12/70) (SD 0.036, figure 1A). After adjustment for case-mix, there was still a large variation between hospitals: the adjusted AL rates per hospital ranged from 0 to 17% (SD 0.033). The reduction in variation after adjustment for case-mix was not statistically significant (p=0.52).

Figure 1

Boxplots presenting the range in hospitals’ anastomotic leakage rates (A) and mortality rates (B). The unadjusted range (left), the range after adjustment for case-mix (centre), and the range after adjustment for case-mix and treatment factors (right) are shown. p Values describe the statistical significance of the reduction in variance (Levene's test); a p value <0.05 was considered statistically significant.

The variance in AL rates significantly decreased after including treatment factors in the adjustment model (p<0.01). Case-mix and treatment-adjusted AL rates varied from 0 to 12% (SD 0.024).

For 60% of the hospitals (50/83), the unadjusted AL rate was similar to the case-mix adjusted AL rate. In 36% of the hospitals, AL rates slightly increased or decreased by 1%, and in 4% of the hospitals by 2% (figure 2A).

Figure 2

Scatterplots showing the effect of adjustment for case-mix (model 2) and case-mix and treatment factors (model 3) for anastomotic leakage (A) and mortality (B) on an individual hospital level. Each scatter represents a single hospital's unadjusted, case-mix adjusted, and case-mix and treatment adjusted rate. On the x-axis, hospitals are ranked according to their unadjusted anastomotic leakage or mortality rate.

For 75% of the hospitals (63/83), the unadjusted AL rate changed after adjustment for treatment factors by at least 1%; for 32% of the hospitals, the unadjusted rate changed by more than 3%; and for 10% of the hospitals, the rate changed by more than 5%.

Although hospital variance decreased after adjustment for case-mix and treatment factors, there was still variability between hospitals as a likelihood ratio test showed that the variance of the random effects was statistically significant in all models.

Postoperative mortality

Hospitals’ unadjusted mortality rates ranged from 0 to 10% (SD 0.017). The variance in postoperative mortality significantly decreased after case-mix adjustment (p<0.01) (range 0–6%, SD 0.012, figure 1B).

The variance in postoperative mortality rates slightly increased (range 0–6%, SD 0.013) after including treatment factors in the adjustment model, although not statistically significant (p=0.81).

For 84% of the hospitals (70/83), the unadjusted postoperative mortality rate changed after adjustment for treatment factors by at least 1%; for 24% of the hospitals, the unadjusted rate changed by more than 3%; and for 6% of the hospital, the rate changed by more than 5% (figure 2B).

Adjustment for treatment factors had a slight effect on two hospitals compared with the case-mix adjusted mortality rate. In these hospitals, the case-mix adjusted mortality rate changed by 1% after adjustment for treatment factors.

Hospital variability in postoperative mortality was still significant after adjustment for case-mix and treatment factors, as a likelihood ratio test showed that the variance of the random effects was statistically significant in all models.

Discussion

The present study suggests that ‘AL rate’ is an outcome indicator for measurement of surgical quality of care that is relatively independent of differences in case-mix between hospitals. We found a large variation in AL rates between Dutch hospitals. This confirms the ability of this outcome indicator to be discriminative. In contrast to ‘postoperative mortality’, the observed AL rates of hospitals evaluated in our study could not be explained by differences in case-mix. In addition, we found that the influence of treatment factors on the variation in AL rates was substantial. These findings imply that AL rates may be much more related to treatment factors and in-hospital care processes than to characteristics of the patient population treated in a certain hospital. AL rates may therefore be a good reflection of the quality of care provided.

Outcome measures

Optimising surgical outcomes can be seen as ‘the bottom line’ of what surgeons do, and outcome indicators have the advantage that they have ‘face validity’ for surgeons and their patients. Also, measurement in itself may improve surgical outcomes, as suggested by the so-called Hawthorne effect.4 As shown in our study, outcome indicators can present meaningful differences between hospitals. However, there was still significant variability in AL and mortality rates after adjustment for case-mix factors and treatment factors in our study. This suggests that there are other characteristics of the hospital, its staff and the care they deliver that may explain the observed differences. Although outcomes of care are important, process and structure information is essential to identify which area is susceptible for innovation. Therefore, adopting the Donabedian paradigm,50 a balanced indicator set needs to include information on structures, processes and outcomes.

Limitations

The results presented in this study should be interpreted in the light of some important limitations. First, despite the fact that most patient-related risk factors were available in the database of the DSCA, it lacked data on some important host-related factors, such as smoking, alcohol consumption, nutrition status and preoperative leukocytosis. Although unlikely, it is possible that a strong case-mix adjustment model for AL could have been made if those four missing factors were available from the dataset.

Also, high-risk patients according to the surgeons’ preoperative risk judgment or patients with impaired continence at baseline may not have been selected for a primary anastomosis and therefore excluded, which may have caused a potential selection bias. It is not exactly clear how these differences in patient selection might affect the between-hospital comparisons.

Moreover, due to a lack of clear agreements on definitions, the factors we used may not have been identical to the ones found in the literature.

Although we found that case-mix adjustment does not seem to play a large role when comparing hospitals’ AL rates, there are some limitations to using it as an outcome indicator. It may unintentionally lead to the perverse incentive of aiming for the lowest possible AL rate by constructing more end colostomies or defunctioning stomas. This defensive attitude would not immediately contribute to increased quality of care as a surgeon or clinic that has zero AL rates at the cost of constructing defunctioning stomas or end colostomies in all patients would not be regarded as providing the best practice. Obviously, AL rates are only calculated for patients in whom an anastomosis has been created. Therefore hospitals with lower rates of patients with anastomoses could automatically have better scores without providing better quality of care, as the stoma itself may cause morbidity, lead to increased readmissions51 ,52 and may be associated with morbidity at the time of surgical removal of the stoma.53 In reality, there is probably an optimum percentage of defunctioning stomas and end colostomies to be constructed, and AL rates should always be seen in the light of these percentages. However, the exact optimum is unclear and it may vary among different surgeons or clinics. Auditing programmes like the DSCA may help to clarify in what range this optimum should be. A composite quality measure might be a solution; that is, a metric which includes whether or not AL occurred, creation of a defunctioning stoma or end colostomy, readmission or mortality. Patient-reported outcomes are of additive value in this context. The choice between an anastomosis with or without a defunctioning stoma or an end colostomy can and should always be influenced by patient preferences.

Improvement of outcomes

When AL is used in hospital comparisons, it should be under the condition that practices with higher AL rates have the opportunity to improve their performance. Unfortunately, the actual cascade of factors resulting in AL still remains a ‘black box’. Our findings suggest that this black box consists of factors that represent multiple elements of the care processes taking place within a hospital. Peroperative factors, such as blood loss and duration of the operation, have been described as important predictors for AL by several authors.9–13 Longer duration, more blood loss than anticipated, an increased anastomotic strain and limited vascular supply at the anastomotic sites may be a proxy for a more complicated procedure, suggesting that AL rates might be related to surgical technical skill and experience. Additionally, factors more related to perioperative care than to surgical skill, such as oliguria during the operation, are also said to enhance the risk of leakage.54

The ultimate challenge for outcome researchers is to understand the complex clinical mechanisms that lead to success or failure, so that the excellence of best practices can be transferred to all hospitals performing these procedures.

Definition of AL

Comparison of AL between hospitals also requires the use of standard definitions and methods of measurement of AL. However, it has been stated before that the definition of AL varies; a systematic review by Bruce et al15 found 56 separate definitions for AL used in the literature. A valuable feature of an audit registration system is that it applies one definition that is used by all participants. In the DSCA, only clinically apparent leaks requiring reintervention have been registered, and a distinction has been made between radiological and surgical reintervention. Further (international) agreement on a standard definition that is valid and reliable, and can distinguish between clinical minor and major anastomotic leaks are explicitly important when using AL as an outcome indicator.

Conclusions

Hospital variation in AL rates is relatively independent of differences in case-mix. Differences in treatment factors contributed more to the variation in AL rates. Further exploration of in-hospital factors may give insight into further improvement possibilities and understanding the multifactorial process that underlies AL. Audit programmes may provide data for targeted visitation of clinics with bad outcomes, and best practices, aiding in identification of the most important areas for improvement.

Acknowledgments

The authors thank all surgeons, registrars, physician assistants, and administrative nurses who registered all the patients in the Dutch Surgical Colorectal Audit, and the Dutch Surgical Colorectal Audit group (E H Eddes, W A Bemelman, R M van Dam, E van der Harst, J H J M van Krieken, J Meijerink, M Elferink, B Groot-Koerkamp, F Ferenschild, D Boerma, E de Graaf, V E Lemmens, E R Manusama, H J T Rutten, C J H van de Velde), and the methodological board for their advice.

References

Footnotes

  • Contributors Study concept: HS, DH, JWD, KH, TW, TK, MW; study design: HS, DH, MW, MF; data acquisition: TK, KH, NvL, TW, RT; quality control of data and algorithms: DH, NvL, MF; data analysis and interpretation: HS, JWD, MW; manuscript preparation: HS, DH, MW; manuscript editing: DH. NvL, MtB, TK, KH, TW, JWD, RT, MW; manuscript review: DH. NvL, MtB, TK, KH, TW, JWD, RT, MW.

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.