Outcomes—a promising way forward for the NHS?

The utility of outcome indicators in identifying serious quality failures is often highlighted. For example, Spiegelhalter and colleagues demonstrated that risk-adjusted monitoring of outcomes could have led to earlier detection of two notable failures of care in the NHS: the unwarranted paediatric cardiac surgery deaths at Bristol Royal Infirmary and the murders of elderly patients by GP Harold Shipman.6 However, both they and others7 8 also highlight the challenges entailed, including whether routinely available outcomes data has adequate sensitivity and specificity to detect such egregious outliers in order for ongoing monitoring to be cost effective. On the other hand, the use of outcome measures for the more general purpose of benchmarking performance to drive quality improvement across the system is both increasingly feasible and likely to be of greater value to the NHS as a whole.

Outcome measurement is a valuable and widely used tool for measuring the quality of healthcare services. Cardiothoracic surgery has led the field internationally in outcome monitoring. The Society for Cardiothoracic Surgery in Great Britain and Ireland (SCTS) reported that cardiac bypass surgical mortality fell by 35% over 7 years,9 and the UK's results are better than the European average. Claims that outcome measurement is unaffordable in austere times have been refuted, with the SCTS stating that the £1.5 million outlay on data collection has led to quality improvements and a £5 million saving in bed days for coronary artery bypass operations alone.10

Evidence from the US Department of Veterans Affairs' (VA) National Surgical Quality Improvement Program (NSQIP) also demonstrates significant improvements in mortality and morbidity following the systematic use of prospective risk-adjusted outcome measures.11 This programme entailed comparative assessment across multiple institutions of postoperative outcomes for several surgical subspecialties. Extension of NSQIP to general and vascular surgery in private sector hospitals was also associated with reductions in adverse postoperative outcomes.12

The NHS outcomes framework aims to make transparent and quantifiable outcome improvement (such as that demonstrated by professionally-led initiatives like those cited above) the norm in the healthcare system. It will be underpinned by the quality standards under development by the National Institute for Health and Clinical Excellence (NICE), covering 12 topics currently but to be extended to 150. Derived from the best available evidence, these standards are markers of high quality, cost-effective patient care for the treatment of different conditions. They are designed to put quality at the heart of the NHS agenda, in the expectation that they will drive improvements in clinical practice and therefore outcomes.

Placing healthcare outcomes in an international context, as the NHS outcomes framework does, is also undoubtedly useful for targeting policy to areas where England compares unfavourably—as, for example, in cancer survival rates. Professor Sir Mike Richards, the National Clinical Director for Cancer, notes that outcome tracking could help us to reach the European average by 2015, saving 5000 lives, with the possibility of saving 10 000 if we could match rates in Australia, Canada and Sweden.13

Established by the government in response to criticisms of the reforms, the Future Forum's review endorsed the outcomes framework as “a powerful new accountability framework for the NHS… designed to focus the whole system from top to bottom on the outcomes achieved for patients”.14

Does the outcomes framework therefore augur well for the NHS?

Outcomes—what else is needed?

The outcomes framework is accompanied by separate outcomes frameworks for assessing performance in public health and social care, which poses risks for integrating care across preventive, health and social care services. The importance of integrated services to address the growing prevalence of chronic and co-morbid conditions, and the cost pressures to reduce hospital admissions through more effective preventive and community care, have been highlighted by many.15 The NHS has historically not performed well in these areas16 and the delineation of roles and accountabilities by having separate outcomes frameworks mitigates against these goals. Likewise, separate accountability frameworks for public health (the responsibility of local authorities) and healthcare (the responsibility of new GP-led clinical commissioning groups) could exacerbate the divide between treatment and prevention services. Public health, prevention and reducing health inequalities are core functions of the NHS, with the potential for general practice to play a key role via both its commissioning and provider roles. The Future Forum echoes these concerns, noting the need for more integration and “a coherent system-wide approach to improving and protecting the public's health”.14

The focus on outcomes itself has been questioned, with many noting its inconsistency with the evidence.17 18 The relative merits of process and outcome measures have been debated for years19–21 with broad consensus in support of Donabedian's structure/process/outcome measurement model, as reflected in most conceptual frameworks used internationally for assessing healthcare system performance.22–24 The limitations of outcome measures—for example, the time lag between intervention and outcome, adequate adjustment for case-mix and severity, impact of factors unrelated to the quality of care, and lack of pointers to action—are well known. Thus for example, cancer screening and prompt access to cancer care are more useful markers of contemporaneous healthcare performance than cancer survival.

Many outcome measures (other than perhaps measures of safety and patient experience where the association is more direct) reflect the impact of healthcare but also of co-morbidities and wider determinants such as socioeconomic factors and lifestyles. Outcome measures are therefore most frequently applied in the context of secondary care, generally surgery, where the link between intervention and outcome is more direct and concurrent, and outcomes are amenable to risk adjustment, frequently aided by the availability of clinical audit data. There are few examples of robust outcome measures for primary and ambulatory medical care where such conditions do not apply. Outside of cardiac surgery and some similarly specialised areas, robust risk-adjusted outcome measures, and the clinical data required to derive them, are hard to come by—for example, for common general medical conditions like heart failure and pneumonia. Furthermore, most evidence-based quality improvement measures—such as those in the Quality and Outcomes Framework and the NICE quality standards—are oriented towards care processes rather than outcomes. As NICE states, “at present there are limited health outcome measures that can be used as quality measures”.

The national clinical audits (eg, stroke, cancer, renal, diabetes) provide a rich data source for developing clinically meaningful process and risk-adjusted outcome measures; some are in use for monitoring and quality improvement purposes, demonstrating the value of supplementing outcomes with process data. But incomplete coverage has limited the wider use of national audit data and the audits do not cover all aspects of healthcare. This void has attracted the use of measures based on hospital administrative data, such as hospital standardised mortality ratios (HSMRs), the sensitivity and specificity of which remains contested and subject to ongoing debate that is testament to the difficulties of using such measures as attributable risk-adjusted outcomes.25–29

An exclusive focus on healthcare outcomes also neglects critical dimensions of healthcare performance—such as timeliness, access and efficiency. These characteristics, valued by patients and the public, are components of many performance frameworks used internationally, and often reflect care processes rather than outcomes. This is already leading to some rethinking: while the clinical community generally welcomed the scrapping of waiting time targets, signs of rising waiting times have led the Prime Minister to announce the reinstatement of some targets.30 The outcomes framework identifies the need to address inequalities in healthcare, but equity is not an explicit dimension of the framework. Although health inequalities continued to widen under the Labour government, its relentless focus on reducing inequalities through targeted NHS and cross-government action is conspicuously lacking in the new strategy. Even when outcomes are used, it is vital to keep in mind which outcomes matter to patients; they may prefer functional outcomes, which are harder to come by, to mortality rates.

The framework also gives no indication of what constitutes acceptable improvement levels on the 51 different indicators included in it. As the financial freeze hits the NHS, public accountability for £120 billion will require parameters of acceptable performance to be defined. Will, for instance, the fall in cancer and heart disease mortality and improvement in cancer survival in the pre-coalition period need to be maintained or accelerated?

Operationalising the outcomes framework—the challenges

Many challenges lie ahead in implementing the outcomes framework. Much will depend on how it is operationalised in the new NHS via the new GP-led clinical groups responsible for commissioning healthcare. The high level outcome goals will need to be translated into locally implementable structure, process and intermediate outcome measures that offer prospects of delivery on the overarching goals. For clinicians on the ground, these delivery issues will be important. The culture change envisaged by architects of the framework depends on staff having locally implementable and quantifiable measures, which will directly show the difference that modifying their practice and auditing is making.

The NICE quality standards are expected to drive the commissioning process. Most of the underpinning measures identified by NICE are process measures which, if implemented, should improve outcomes. However, this cannot be guaranteed and most NICE measures cannot be derived from currently available datasets. So it is likely that clinicians will face demands on their time for greater data collection for audit purposes, possibly requiring investment in IT infrastructure to support such data collection.

Furthermore, the wider, practical implications of measuring outcomes of NHS care need consideration. The IT infrastructure will need to be adapted to the challenges that lie ahead for the NHS, and this will require investment. For example, if more care is to be delivered in the community, and given the growing prevalence of long-term conditions and co-morbidities, a robust IT system that straddles GP practices, other providers, clinical commissioning groups and local agencies (and as exists in some areas currently) will be essential. GPs will want to monitor use and outcomes of secondary care services for their patients, and data on post-operative follow-up will support not just meaningful outcome measurement but also revalidation and learning for surgeons.

Ways forward

A national outcomes-oriented accountability framework for the NHS has many attractions: it is simple and intuitively appealing to many stakeholders. All healthcare systems aspire to improved outcomes and embedding the NICE quality standards in the NHS offers real potential for improvements in the delivery of evidence-based care. But, as we have argued, the framework does not of itself guarantee delivery on the political aspirations for world class health outcomes. Furthermore, while outcome measurement is very useful for monitoring and accountability for a publicly funded service, an exclusive focus on outcomes is not a sufficiently comprehensive basis for judging the performance of an entire healthcare system. Accountability mechanisms also need to incorporate a focus on other dimensions such as access, equity, efficiency and integrated care, to meet the challenges that lie ahead for the NHS. Finally, the framework will need considerable ‘deconstruction’ in its application locally into actionable structure and process measures that will support quality and outcomes improvement.

A pragmatic approach going forward, especially given the financial constraints facing the NHS, is to ensure that national health, public health and social care goals are closely aligned. Simultaneously, the supporting accountability mechanisms, including those used locally by the clinical commissioning groups, will need to be underpinned by a judicious blend of structure, process and outcome measures that the NHS and local government can work jointly to achieve.