The National Health Service in England is moving away from targets based on processes of care and focusing on patient outcomes. This vision is operationalised in the recently published NHS Outcomes Framework, which includes the generalised use of Patient Reported Outcomes (health status and quality of life) as measures of population health at the provider level. This is the first time that such a bold initiative is attempted in the UK and it is not without risks. In this article we elaborate on our experience on the use of Patient Reported Outcomes and identify challenges and likely implications of this approach and suggest less disruptive alternatives.
Statistics from Altmetric.com
In December 2010, the UK Department of Health released the measures that would be used to assess the performance of the NHS. The NHS Outcomes Framework sets out a vision for a performance model based on health outcomes rather than processes of care.1 The selection of indicators is consistent with the progressive incorporation of outcome measures, including patient reports,2 such as the systematic collection of preoperative and postoperative Patient Reported Outcomes Measures (PROMs) in selected elective surgical procedures3 and the national GP Patient Survey.4
The Department of Health aims to capture the change in mean scores from year to year on a short health status measure, EQ-5D, as one of its key measures of quality in the NHS,5 with scores adjusted for case mix or clinical complexity. EQ-5D is a widely used generic health status measure, but it has never been used in this way before. Is this a wise choice?
Health status measures have previously been seen as measures of need in health service planning. The proposed use of EQ-5D as a measure of outcome to reflect quality of care is a radical departure which poses four major challenges. Three of these relate to valid interpretations of change scores, and one relates to potential unintended and undesirable consequences which may emerge.
First, we do not know the impact of healthcare on EQ-5D scores. Although there are cross-sectional data on EQ-5D scores at the population level for a range of conditions,6–9 we lack fundamental information from longitudinal studies on the impact of healthcare on these scores or whether they are actually amenable to healthcare at all.10 Indeed, a study published only last year claims to be the first to demonstrate a longitudinal association between improved care and improved self-reported health.11 Furthermore, it is also well established that the association between processes and outcomes of care is generally small.12 13 In short, we do not know if improving healthcare improves self-reported health status at the population level, and there are examples of precisely the reverse occurring. It is easy to think of situations in which earlier diagnosis and more aggressive treatment may in the short term increase the impact that a chronic disease has on a person's life.
Second, we do not know how much natural variability there is in EQ-5D scores over time, especially when carried out through a postal survey as proposed (the GP Patient Survey). The Department of Health has already had to retreat from using this survey as a basis for paying GPs because of a problem that resulted from a payment formula not taking sufficient account of random variation.14
The third methodological hurdle has to do with the selection of case-mix variables. It is not clear how helpful it will be to adjust for the prevalence of specific long-term conditions and/or their severity or indeed how such conditions might be selected. Rigorous testing is required before EQ-5D can be used with confidence as a measure of outcome of healthcare at the population level.
The most worrying challenge for health policymakers, however, is related to unintended consequences and perverse incentives that may arise. By focusing on changes from year to year, there is the risk of creating incentives for providers to improve short-term outcomes at the expense of longer-term gains. At present, it is very difficult to predict how this would alter the perceptions of purchasers and providers of the benefits of particular interventions. The Department of Health has announced a move away from ‘targets’ and suggests that data on health status will only be used to inform decisions of commissioners rather than being used for performance management. Although it is possible that providers will use these measures for self-driven quality improvement, the available evidence on the impact of these measures on clinical practice is sparse for population-level measures and non-conclusive for individualised measurement.15 Past experience suggests that whatever measures are used to assess NHS care, they are bound to be used in some way to manage performance. In addition, there is a risk, depending on how health status scores are used, that GP practices may become reluctant to enrol patients in poor health, a phenomenon known as cream skimming.
In summary, there are substantial uncertainties about the appropriateness of the proposed use of the EQ-5D, and the suggested timeframe risks creating the wrong incentives. There is no doubt that the increasing focus on outcomes of care reflects a well intended approach towards aligning the priorities of the providers with those of the patients. It is a step in the right direction, and there are a number of factors to support it. Indeed, measuring outcomes and processes of care may uncover the ineffectiveness of some care processes in spite of their demonstrated benefits under experimental circumstances, and this may help identify problems with implementation. The advantage of a performance model including outcomes is that it explicitly makes them a priority, opening the door to embedded evaluations that are patient centred (health outcomes) and health system centred (system outputs). However, as demonstrated by the problems identified for the proposed use of the EQ-5D, substantial challenges lie ahead if outcomes are to be successfully integrated into the routine evaluation of the impact of healthcare at the population level. If these problems are not addressed, there is a risk that any issues potentially arising from the use of health status measures may be considered as a failure of the measurement model itself rather than a problem in the construction and interpretation of specific indicators.
The experience of the National Committee on Quality Assurance in the USA might prove useful. Quality indicators in the US HEDIS set undergo extensive testing and public comment before they are reviewed by an outside advisory group, the Committee on Performance Measurement. They are then included for 1 year only before review and final approval.16 Similar caution should be used before deciding how our health service should be assessed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.