Money and activity versus quality: a false dichotomy

In England, the NHS has been slower to grasp the challenge of measuring quality. Until recently, policy makers have been content to pride themselves on having moved to a position in which quality improvement is a central goal of the system. Laudable though this has been, the dangers of espousing quality as a goal strategically without considering how to operationalise it properly (including measurement) have been evident in spectacular failures in standards of care when the much more easily quantifiable measures—money and activity—were the true priority for managers of the system.4 ,5

This issue has persistently rankled with professional staff delivering front-line care. No matter how often the language of quality and safety is spoken by those assessing performance, the true lingua franca of healthcare is financial. Many clinicians in Britain remain unshakeable in their belief that what really matters to patients or themselves is not the same as what influences those running the service. The portrayal of this as a difference in values further deepens the rift in perceived perspectives. The dysfunction has arguably been worsened by the NHS policy of Payment by Results (PbR)6 introduced in 2003. In most cases, this links payment to volume of activity, although there are a growing number of ‘Best Practice Tariffs’ incorporating quality measures, such as rapid access to a CT scan for those who have suffered a stroke.7 PbR has made clinicians aware of the way in which data derived from patient records are linked to financial reward for their institutions. Meanwhile, the absence of a comparable set of powerful data on the quality of care provided—data that are universally believed and trusted—sustains this divide between managerial and clinical cultures.

NHS hospital organisations are now required to produce a set of quality accounts together with their financial report, but the level of parity the board places between them remains to be seen. The argument that the business plan of a hospital and the quality plan might be one and the same document can bring a glimmer of enlightenment to even the most hardened managerial traditionalist, but it has not yet translated to a paradigm shift in the concept of accountability and how performance is judged.

Clinically curated versus administratively derived data

The successful development of widely accepted and extensively used quality measures in the NHS has been limited to particular fields of care. The Myocardial Ischaemia National Audit Project (MINAP)8 is a prominent example. All hospitals in England and Wales that admit patients with acute coronary syndromes provide data. This has allowed a rich and detailed description of the quality of care in different centres and over time. It has allowed services to be held to account for standards set in the National Service Framework for coronary heart disease.9 It has been embraced clinically and managerially, and other initiatives involving rigorous collection of specified quality data in heart disease have enabled public reporting of service performance as well as non-emotive public debate about variations in standards of care.10 However, the number of truly clinically led national data collection systems is limited and restricted to major conditions.11

By contrast, the derivation and use of a high-level summary index of quality, such as the Hospital Standardised Mortality Ratio, from routinely available data has been dogged by controversy.12 Views have been polarised between those who point to the measure's indispensability in uncovering hitherto hidden problems in a service, and those who consider that they should defend, to the hilt, reputations from a statistic that they consider completely flawed. The time-honoured middle-ground position, that such a measure is only a starting point for further investigation, has held little sway in the heat of recent public discourse about measuring the quality of a hospital's services.

The main source of NHS information on hospital patients, from which this and similar summary measures of quality are derived is Hospital Episode Statistics. This system has been criticised for substandard data quality.13 ,14 The quality of routinely available data is not a good reflection of clinical practice, and clinicians show little interest in it. Indeed, in one study, only 8% of clinicians participated in the validation of clinical coded data.13 Poor quality data lead to lack of confidence in them, and ultimately rejection of findings derived from them. A project undertaken by the Royal College of Physicians of London and the Department of Health in England sought to secure greater clinical engagement in routinely available data systems with the aim of improving their quality, use and relevance.15 It found that clinicians were wary that such data might be used for judging them because the scope did not encompass outpatients where much of their work was based, and because modern care is delivered by teams so that consultant-specific quality measures are not reliable.16

The strength of clinical specialty or condition-specific systems, like MINAP, that capture quality measures is that they are mainly clinically curated. Thus, the data selected for inclusion are relevant and outcome orientated; capture of data is comprehensive, and they are less disputed by outliers as often happens with data gathered under management auspices. When the data standards have been set by your peers, it is harder to wave away, or excuse, inconvenient statistics about your service.

NHS system reforms require good data

A commitment to making quality and safety the ‘organising principle’ of the NHS was made by Prime Minister Brown's Government in its White Paper, High Quality Care for All.17 Background work for this strategy found that the highest performing clinical teams used quality data routinely.18 In one of the leading hospitals in the USA, on one quality-ranking scheme, the approach to gaining full clinical engagement has been to make the use of data a credible ‘scientific’ endeavour rather than an activity required by management (Pronovost P; personal communication). Creating a clinical culture in which such ‘dataphilia’ among clinicians in the NHS is commonplace is a major challenge, but if it could be done, there would be major gains. The reasons why it is not so, currently, are again deep seated and include: limited exposure in undergraduate and postgraduate medical education programmes to quality-of-care concepts; a lack of confidence in the validity of routine data; the absence of professional leadership in embedding it as a core component of good clinical practice; and the low value of health services research (using such data) in career advancement compared with clinical and molecular research.

Meyer et al3 call for greater selectivity and for counting things that matter. They argue for metrics that serve the needs of end users (patient, families, payers) for accountability of performance and for judging quality and value while achieving parsimony in the number and type chosen. It is important that there are such system-wide indicators that provide insight into the quality and safety of care being provided, whether that ‘system’ is the whole provision of care in a country or that delivered in a hospital or primary care service. However, it is equally important that front-line clinical teams also have data that they use to compare themselves with the best, and to assess whether they are improving over time. Those data will encompass many more measures of quality and will relate closely to the specific context of the area of care. Health policy makers are not always sure whether to encourage clinicians to have as much data as they feel they need in order to evaluate their services. There is also uncertainty on how much of such data should be extracted for management oversight, or rolled into summary measures for monitoring system performance. Moreover, there is a paucity of good methodological work on how best to aggregate information from a diversity of quality measures.

Major reforms to the NHS in England, now being implemented, create an urgent need for good data in two areas. First, with plans to devolve most decisions about the planning and funding of care for populations to local level in entities called Clinical Commissioning Groups, the previous management bodies that controlled these functions will disappear. Accountability for the delivery of provider-contractual requirements, for assessing return on public investments and for seeking equity of care, will only be possible through data. Second, the new system relies on choice by patients. This too will require data.

The coalition government that came to power in 2010 set out these reforms in its White Paper, Equity and Excellence: liberating the NHS.19 This acknowledges the importance of data on quality and sets out proposals for an NHS Outcomes Framework covering five outcome domains (premature death, quality of life for chronic disease, recovery from illness or injury, experience of care, safety), and a comprehensive suite of standards to support the framework. However, the focus on outcome over process metrics,20 the centrally defined nature of the framework, and the lack of widespread clinical consensus have led to criticism. A library of some 150 standards is to be developed over the next 5 years by the National Institute for Health and Clinical Excellence, which may generate a more clinically plausible, if not lean, set of standards.21

After a series of horror stories about staff attitudes towards care, often when looking after elderly or vulnerable patients,22 the measurement of patient experience has gained a renewed focus within the NHS. Although there are existing detailed national random surveys of experience, asking a broad array of questions, the English NHS is now pushing the notion of a simple, single question about experience borrowed from a consumer notion of customer service: would you recommend to family and friends the service you have received? However, in this drive towards simplicity, there have been further voices of discontent. In reviewing the approach, the Picker Institute suggested that many people thought of asking a cancer survivor whether they would recommend their service—a potentially insensitive question to ask23—demonstrating that simplicity alone is not always acceptable.

Data that measure the quality of healthcare are needed for accountability, consumer choice and quality improvement. In their development, the fundamental tension of performance measurement comes to the fore. Metrics must be acceptable to clinicians, collectable from management systems and understandable by the public. A simple triad that is hard to reconcile.