Article Text

other Versions

PDF
Displaying random variation in comparing hospital performance
  1. A M van Dishoeck1,
  2. C W N Looman1,
  3. E C M van der Wilden-van Lier2,
  4. J P Mackenbach3,
  5. E W Steyerberg1
  1. 1Department of Public Health, Center for Medical Decision Making, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
  2. 2Directorate of Patient Care, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
  3. 3Department of Public Health, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
  1. Correspondence to A M van Dishoeck, Department of Public Health, Center for Medical Decision Making, Erasmus MC, University Medical Center Rotterdam, Room AE-138, PO Box 2040, 3000 CA Rotterdam, The Netherlands; a.m.vandishoeck{at}erasmusmc.nl

Abstract

Introduction The role of transparency in quality of care is becoming ever more important. Various indicators are used to assess hospital performance. Judging hospitals using rank order takes no account of disturbing factors such as random variation and case-mix differences. The purpose of this article is to compare displays for the influence of random variation on the apparent differences in the quality of care between the Dutch hospitals.

Method The authors analysed the official 2005 data of all 97 hospitals on the following performance indicators: pressure ulcer, cerebro-vascular accident and acute myocardial infarction. The authors calculated CIs of the point estimate and the simulated CIs of the ranks with bootstrap sampling, and visualised the influence of random variation with three modern graphical techniques: forest plot, funnel plot and rank plot.

Results Statistically significant differences between hospitals were found for nearly all performance indicators (p<0.001). However, the CIs in the forest plot revealed that only a small number of hospitals performed significantly better or worse. The funnel plot provides a representation of differences between hospitals compared with a target value and allows for the uncertainty of these differences. The rank plot showed that ranking hospitals was very uncertain.

Conclusion Despite statistically significant differences between hospitals, random variation is a crucial factor that must be taken into account when judging individual hospitals. The funnel plot provides easily interpretable information on hospital performance, including the influence of random variation.

  • Performance indicators
  • performance measurement
  • hospitals
  • funnel plot
  • random variation
  • assessment
  • healthcare quality
  • organisation
  • outcome
  • quality of care

Statistics from Altmetric.com

Footnotes

  • Funding Internal Erasmus MC grant for Healthcare Research (Mrace).

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.