Article Text
Statistics from Altmetric.com
For many years, our understanding of what ‘quality’ and ‘performance’ mean in health services has been limited to analysing counts of how many patients are treated (admissions, patient visits), how quickly that is done (waiting times) and how often things go wrong (complications, re-admissions, deaths). All of these things are important, but they don't tell us about something of fundamental importance: how have those health services affected health?
In 2009, the UK Department of Health introduced the routine collection of patient-reported outcome measures into the English NHS via the PROMs programme.1 2 In doing so, it became the first healthcare system in the world routinely to collect patients' self-reported health status in the context of normal health services delivery. The scheme initially focused on collecting these data before and after four elective surgical procedures, with plans to extend this across a number of other important areas, including chronic diseases. The PROMs programme should not be seen as part of the latest set of NHS ‘reforms’, because it predates them by some years. Instead, it should be viewed as the latest stage in an attempt to improve how the performance and quality of health services is measured and analysed. Rather than being a step in the dark, it might be better described as an attempt to cast some further light on important areas that are currently very dimly lit or even wholly in the dark.
This policy is a highly ambitious and logistically challenging undertaking—and there is much to learn about how best to analyse, report and use the data in decision making. It is also an opportunity for the NHS to lead, internationally, in good ways of using patients' views about their health to improve healthcare.
Valderas et al3 rightly suggest that we should exercise great caution in introducing any new policy initiative such as this. However, if their critique is intended to refer to the PROMs programme, this suggests some confusion about its aims and what the evidence says about the EQ-5D. Moreover, their pessimistic speculation about the ‘unintended consequences’ and ‘perverse incentives’ of measuring outcomes is not informed by an analysis of the incentives given by the status quo and its actual consequences.
The confusion arises from their use of ‘population’ as a general term to mean any collection of people. This gives them the ability to slide between evidence and arguments relating to quite different kinds of populations. For example, in reading their note, it might be concluded that the aim is to provide a health outcome measure for the whole English population and to see the impact of the NHS on that. As they note, there is no single measure, not just the EQ-5D, which meets that aim. Or is it a population within a general practice, and how the impact of healthcare varies between GPs? Or a population defined by geographical area, socioeconomic group or disease? Again, they are correct that although there are good measures of need based on health status, including the EQ-5D, we do not have evidence that these form good measures of outcome at that level of aggregation.
But the PROMs programme concerns healthcare processes that are, or if they are not should be, directly linked to outcomes of care. The link may be complex, but if it does not exist, then we might wonder why the process is being undertaken at all. Of course short-term outcomes might not reflect long-term outcomes, so we must be careful with our timing of when they are measured. However, that's not an argument for not measuring neither of them.
This confusion may explain the puzzling assertions that ‘we do not know what (the) impact of healthcare is on EQ-5D scores’ and that there is a lack of ‘information from longitudinal studies on the impact of healthcare on these scores’. On the contrary, there is considerable evidence from over two decades of clinical studies using EQ-5D to gauge the impact of healthcare on health. Wailoo et al concluded on the basis of the evidence that it is an appropriate measure in most disease areas.4 That is the reason why, for example, the National Institute for Health and Clinical Excellence recommend the EQ-5D as the preferred instrument for evidence presented to it in appraising the effectiveness and cost effectiveness of new technologies.5 Although the EQ-5D is widely used in general population studies and provides valuable information, both cross-sectional and longitudinal, it is true that its use in that context has not been linked with the specifics of healthcare. It's also worth remembering that the EQ-5D is only one of the measures included in the current PROMs programme—and that the Outcomes Framework,6 similarly, relies on a set of outcomes indicators, not just one measure.
In speculating about unintended consequences and perverse incentives, such as cream skimming, it's important to remember that these apply equally to any measure of performance. It does not matter whether or not it is outcomes related, or patient reported or case-mix adjusted. The problem is certainly not caused by using the EQ-5D instead of any other health outcome measure. That some healthcare organisations and health professionals will try to ‘game’ the system for financial advantage is not an argument not to measure performance. If they will do that in the future, they are doing it now. If we decide not to measure health outcomes, what will be the unintended consequences of that decision, and what perverse incentives will we leave in place?
As usual, when we are thinking about policy initiatives, the most useful approach is to consider both the costs and the benefits. Because the Department of Health and the wider ‘outcomes movement’ are keen to introduce health outcomes measurement into assessment of performance, they have emphasised its potential benefits. The argument for introducing this now and in this particular form is that there is no definitive or demonstrably better form for it, which means that there will always be a means of arguing against introducing anything. If we do that, there will never be any evidence about its consequences. We should see the concerns raised by Valderas et al as a plea to consider a range of possible costs in a future evaluation of the PROMs initiative, not as an argument that it needs to be stopped.
Footnotes
Linked article 000184.
Disclaimer The views expressed here are those of the authors, and do not necessarily represent the views of the NHS South East Coast.
Competing interests Both David Parkin and Nancy Devlin are members of the EuroQol Group, a not-for-profit charity that owns the copyright in the EQ-5D, and Nancy Devlin is Chair of its Executive Committee.
Provenance and peer review Commissioned; internally peer reviewed.