Article Text

Download PDFPDF

What is a performance outlier?
  1. David M Shahian1,2,
  2. Sharon-Lise T Normand3,4
  1. 1Department of Surgery, Center for Quality and Safety, Massachusetts General Hospital, Boston, Massachusetts, USA
  2. 2Harvard Medical School, Boston, Massachusetts, USA
  3. 3Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts, USA
  4. 4Department of Biostatistics, Harvard School of Public Health, Boston, Massachusetts, USA
  1. Correspondence to Dr David M Shahian, Department of Surgery, Center for Quality and Safety, Massachusetts General Hospital, 55 Fruit St, Boston, MA 02114, USA; dshahian{at}

Statistics from

Healthcare performance measurement is a complex undertaking, often presenting a number of potential alternative approaches and methodological nuances. Important considerations include richness and quality of data sources; data completeness; choice of metrics and target population; sample size; patient- and provider-level data collection periods; risk adjustment; statistical methodology (eg, logistic regression vs hierarchical models); model performance, reliability and validity; and classification of outliers. Given these many considerations, as well as the absence of nationally accepted standards for provider profiling, it is not surprising that different rating organisations and methodologies may produce divergent results for the same hospitals.1–6

Outlier classification, the last step in the measurement process, has particularly important ramifications. For patients, it may lead them to choose or avoid a particular provider. For providers, outlier status may positively or negatively impact referrals and reimbursement, and may influence how scarce hospital resources are deployed to address putative areas of concern. Misclassification is probably more common than generally appreciated. For example, partitioning of hospitals (eg, terciles, quartiles, quintiles, deciles) to determine outliers may lead to excessive false positives—hospitals labelled as having above or below average performance when, in fact, their results do not differ significantly from the mean based on appropriate statistical tests.7 ,8

The current study

In this issue, Paddock et al9 address a seemingly straightforward question—what precisely does it mean to be a performance outlier? Using Hospital Compare data, the authors demonstrate an apparently contradictory finding. When directly compared one to another, some individual hospitals in a given performance tier may not be statistically significantly different than individual hospitals in adjacent tiers, even when those tier assignments were made using appropriate tests of statistical significance. For instance, Paddock et al9 show that for each bottom-tier hospital, there was at least one mid-tier hospital with statistically indistinguishable performance. Among …

View Full Text

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles