Article Text

This article has a correction. Please see:

Download PDFPDF

From ‘reckless’ to ‘mindful’ in the use of outcome data to inform service-level performance management: perspectives from child mental health
  1. Miranda Wolpert1,
  2. Jessica Deighton2,
  3. Davide De Francesco2,
  4. Peter Martin2,
  5. Peter Fonagy3,
  6. Tamsin Ford4
  1. 1Child and Adolescent Mental Health Services (CAMHS) Outcomes Research Consortium, Evidence Based Practice Unit, UCL and Anna Freud Centre, London, UK
  2. 2Evidence Based Practice Unit, UCL and Anna Freud Centre, London, UK
  3. 3Research Department of Clinical, Educational and Health Psychology, UCL and Anna Freud Centre, London, UK
  4. 4Medical School, University of Exeter, Exeter, UK
  1. Correspondence to Dr Miranda Wolpert, Child and Adolescent Mental Health Services (CAMHS) Outcomes Research Consortium, Evidence Based Practice Unit, UCL and Anna Freud Centre, 21 Maresfield Gardens, London, NW3 5SU, UK; ebpu{at}annafreud.org

Statistics from Altmetric.com

Introduction

In the recent report on patient safety in the National Health Service (NHS) in England, Don Berwick calls on the NHS to align the necessity for increased ‘accountability’ with the necessity to ‘abandon blame as a tool’ in order to develop a ‘transparent learning culture’.1 Sir Bruce Keogh, Medical Director NHS, and colleagues’ recent analysis of outlier hospitals based on mortality data marks a key step on this journey, but has led to high-profile debate about the risk of possible ‘reckless’ (Sir Bruce Keogh's term) use of data if appropriate parameters are not established.2 ,3 If these and other equivalent proxies for outcomes are to be used safely and effectively to support performance management and quality improvement in the ways envisioned by both Keogh and Berwick, it is crucial to establish clearly agreed operational procedures. Drawing on our experience of collecting and interpreting outcome data in the challenging context of child mental health across the UK, we suggest adoption of a MINDFUL framework involving consideration of multiple perspectives, interpreting differences in the light of current evidence base, focus on negative differences when triangulated with other data, directed discussions based on ‘what if this were a true difference’ (employing the 75–25% rule), use of funnel plots as a starting point to consider outliers, appreciation of uncertainty as a key contextual reality and the use of learning collaborations to support appropriate implementation and action strategies.

Complexities

Any attempt to measure ‘impact’ of a service using a given ‘outcome’ is complex. The Keogh report acknowledges: “two different measures of mortality, HSMR [Hospital Standardised Mortality Ratio] and SHMI [Summary Hospital Level Mortality Indicator] generated two completely different lists of outlier trusts.” This was ‘solved’ by using both lists, but with a suggestion to move to …

View Full Text

Footnotes

  • Miranda Wolpert and colleagues draw on their learning from national outcome data collection across child mental health services in the UK to propose a framework for lead clinicians and funders to collaboratively adopt to ensure effective and safe use of outcome data for performance management and quality improvement across the range of healthcare settings.

  • Contributors MW conceived of and led the writing of this article and acts as a guarantor. JD contributed to the writing and to the plan for the analysis of data. DDF contributed to the data analytic plan and undertook the data analysis. PM contributed to the analytic plan and the data analysis and discussion. PF provided comments and contributions to the content and structure. TF contributed to the writing, to the structure and the data analysis. Information used to prepare the article comes from the CORC dataset.

  • Competing interest MW is a paid director (1 day a week) for CORC; a not-for-profit learning collaboration. TF is an unpaid director of CORC.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles

  • Correspondence
    BMJ Publishing Group Ltd