You are here

Article Text

Article menu
Download PDFPDF
Original research
Ranking hospitals: do we gain reliability by using composite rather than individual indicators?
  1. Stefanie N Hofstede1,
  2. Iris E Ceyisakar2,
  3. Hester F Lingsma2,
  4. Dionne S Kringos3,
  5. Perla J Marang-van de Mheen1
  1. 1 Department of Biomedical Data Sciences, Medical Decision Making, Leiden University Medical Centre, Leiden, The Netherlands
  2. 2 Department of Public Health, Erasmus MC, Rotterdam, The Netherlands
  3. 3 Department of Public Health, Academic Medical Centre, Amsterdam, The Netherlands
  1. Correspondence to Dr Perla J Marang-van de Mheen, Department of Biomedical Data Sciences, J10-S, Leiden University Medical Centre, Leiden 2300 RC, The Netherlands; p.j.marang{at}lumc.nl

Abstract

Background Despite widespread use of quality indicators, it remains unclear to what extent they can reliably distinguish hospitals on true differences in performance. Rankability measures what part of variation in performance reflects ‘true’ hospital differences in outcomes versus random noise.

Objective This study sought to assess whether combining data into composites or including data from multiple years improves the reliability of ranking quality indicators for hospital care.

Methods Using the Dutch National Medical Registration (2007–2012) for stroke, colorectal carcinoma, heart failure, acute myocardial infarction and total hiparthroplasty (THA)/ total knee arthroplasty (TKA) in osteoarthritis (OA), we calculated the rankability for in-hospital mortality, 30-day acute readmission and prolonged length of stay (LOS) for single years and 3-year periods and for a dichotomous and ordinal composite measure in which mortality, readmission and prolonged LOS were combined. Rankability, defined as (between-hospital variation/between-hospital+within hospital variation)×100% is classified as low (<50%), moderate (50%–75%) and high (>75%).

Results Admissions from 555 053 patients treated in 95 hospitals were included. The rankability for mortality was generally low or moderate, varying from less than 1% for patients with OA undergoing THA/TKA in 2011 to 71% for stroke in 2010. Rankability for acute readmission was low, except for acute myocardial infarction in 2009 (51%) and 2012 (62%). Rankability for prolonged LOS was at least moderate. Combining multiple years improved rankability but still remained low in eight cases for both mortality and acute readmission. Combining the individual indicators into the dichotomous composite, all diagnoses had at least moderate rankability (range: 51%–96%). For the ordinal composite, only heart failure had low rankability (46% in 2008) (range: 46%–95%).

Conclusion Combining multiple years or into multiple indicators results in more reliable ranking of hospitals, particularly compared with mortality and acute readmission in single years, thereby improving the ability to detect true hospital differences. The composite measures provide more information and more reliable rankings than combining multiple years of individual indicators.

  • quality improvement methodologies
  • quality measurement
  • continuous quality improvement
  • healthcare quality improvement
  • mortality (standardized mortality ratios)

https://doi.org/10.1136/bmjqs-2017-007669

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Footnotes

    • Contributors PJM-vdM designed the study. SNH wrote the article and carried out the study. PJM-vdM supervised the study and writing of the manuscript. All authors have critically read and modified both the study protocol and previous drafts of the manuscript and have approved the final version. All authors read and approved the final manuscript.

    • Funding This study was funded by ZonMw (10.13039/501100001826) and grant number 516022513.

    • Competing interests None declared.

    • Patient consent Not required.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement We used routinely collected administrative admission data of the Dutch National Medical Registration (LMR) from 2007 to 2012 retrieved from Statistics Netherlands. To use these data, please contact Statistics Netherlands.

    Linked Articles