Displaying random variation in comparing hospital performance
- 1Department of Public Health, Center for Medical Decision Making, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
- 2Directorate of Patient Care, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
- 3Department of Public Health, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
- Correspondence to A M van Dishoeck, Department of Public Health, Center for Medical Decision Making, Erasmus MC, University Medical Center Rotterdam, Room AE-138, PO Box 2040, 3000 CA Rotterdam, The Netherlands; a.m.vandishoeck{at}erasmusmc.nl
-
Contributors JPM and ECMvdW-vL had the original idea for the study, and EWS developed the study design. AMvD gathered the data, and CMNL analysed the data. AMvD wrote a first draft of the paper. All authors contributed to further drafts. Parts of this research have been published in the Dutch language in the Nederlands Tijdschrift voor Geneeskunde (van Dishoeck AM, Looman CM, van der Wilden-van Lier EC, et al. Outcome assessment in hospitals. The influence of insecurity (in Dutch). Ned Tijdschr Geneeskd 2009;153:804e11).
- Accepted 6 September 2010
- Published Online First 12 January 2011
Abstract
Introduction The role of transparency in quality of care is becoming ever more important. Various indicators are used to assess hospital performance. Judging hospitals using rank order takes no account of disturbing factors such as random variation and case-mix differences. The purpose of this article is to compare displays for the influence of random variation on the apparent differences in the quality of care between the Dutch hospitals.
Method The authors analysed the official 2005 data of all 97 hospitals on the following performance indicators: pressure ulcer, cerebro-vascular accident and acute myocardial infarction. The authors calculated CIs of the point estimate and the simulated CIs of the ranks with bootstrap sampling, and visualised the influence of random variation with three modern graphical techniques: forest plot, funnel plot and rank plot.
Results Statistically significant differences between hospitals were found for nearly all performance indicators (p<0.001). However, the CIs in the forest plot revealed that only a small number of hospitals performed significantly better or worse. The funnel plot provides a representation of differences between hospitals compared with a target value and allows for the uncertainty of these differences. The rank plot showed that ranking hospitals was very uncertain.
Conclusion Despite statistically significant differences between hospitals, random variation is a crucial factor that must be taken into account when judging individual hospitals. The funnel plot provides easily interpretable information on hospital performance, including the influence of random variation.
- Performance indicators
- performance measurement
- hospitals
- funnel plot
- random variation
- assessment
- healthcare quality
- organisation
- outcome
- quality of care
Footnotes
-
Funding Internal Erasmus MC grant for Healthcare Research (Mrace).
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.
