Article Text

Enhancing feedback on performance measures: the difference in outlier detection using a binary versus continuous outcome funnel plot and implications for quality improvement
  1. Laurien Kuhrij1,
  2. Erik van Zwet2,
  3. Renske van den Berg-Vos1,3,
  4. Paul Nederkoorn1,
  5. Perla J Marang-van de Mheen2
  1. 1Department of Neurology, Amsterdam University Medical Centres, Amsterdam, The Netherlands
  2. 2Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, Zuid-Holland, The Netherlands
  3. 3Department of Neurology, OLVG, Amsterdam, Noord-Holland, The Netherlands
  1. Correspondence to Dr Perla J Marang-van de Mheen, Department of Biomedical Data Science, Leiden Universitair Medisch Centrum, Leiden, the Netherlands; p.j.marang-van_de_mheen{at}lumc.nl

Abstract

Background Hospitals and providers receive feedback information on how their performance compares with others, often using funnel plots to detect outliers. These funnel plots typically use binary outcomes, and continuous variables are dichotomised to fit this format. However, information is lost using a binary measure, which is only sensitive to detect differences in higher values (the tail) rather than the entire distribution. This study therefore aims to investigate whether different outlier hospitals are identified when using a funnel plot for a binary vs a continuous outcome. This is relevant for hospitals with suboptimal performance to decide whether performance can be improved by targeting processes for all patients or a subgroup with higher values.

Methods We examined the door-to-needle time (DNT) of all (6080) patients with acute ischaemic stroke treated with intravenous thrombolysis in 65 hospitals in 2017, registered in the Dutch Acute Stroke Audit. We compared outlier hospitals in two funnel plots: the median DNT versus the proportion of patients with substantially delayed DNT (above the 90th percentile (P90)), whether these were the same or different hospitals. Two sensitivity analyses were performed using the proportion above the median and a continuous P90 funnel plot.

Results The median DNT was 24 min and P90 was 50 min. In the binary funnel plot for the proportion of patients above P90, 58 hospitals had average performance, whereas in the funnel plot around the median 14 of these hospitals had significantly higher median DNT (24%). These hospitals can likely improve their DNT by focusing on care processes for all patients, not shown by the binary outcome funnel plot. Similar results were shown in sensitivity analyses.

Conclusion Using funnel plots for continuous versus binary outcomes identify different outlier hospitals, which may enhance hospital feedback to direct more targeted improvement initiatives.

  • audit and feedback
  • quality improvement
  • performance measures
https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

View Full Text

Statistics from Altmetric.com

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Contributors First and last author designed the study and wrote the first draft of the manuscript. Second author developed both funnel plots. First and last author performed the additional analyses. All authors provided critical feedback and helped shape the manuscript. All authors approved final version for publication.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available on reasonable request. We used data from the Dutch Acute Stroke Audit, a nationwide clinical audit. On reasonable request, the data are available. The statistical analysis plans are added as a supplement.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.