Ensuring accountability and improving quality are two of the most significant challenges facing health systems around the world. The public release of comparative standardised information on quality in the form of “report cards” represents one suggested solution to these complex problems.¹ Report cards are not new—Florence Nightingale produced a report comparing the mortality rates of London teaching hospitals in 1863²—but nevertheless they have been embraced with great enthusiasm in many developed countries in recent years. In the UK, for example, the introduction of report cards is one of the specific recommendations in a plan to modernise the National Health Service.³

Given this enthusiasm, it is perhaps surprising that we know so little about the uses, benefits, and risks of publicising comparative information. Most of the experience and evidence in this field comes from the USA where report cards have been a prominent feature for the last 15 years. Two papers in this issue of QHC describe the current state of play in the USA and help to develop our understanding of whether and how report cards fulfil the ambitious claims made of them.

It may seem self-evident that the general public should be the primary audience for the “public” release of comparative information. Not so, it would appear, according to Schneider and Lieberman.⁴ Even in the consumer orientated USA where users have expressed a desire for the information in principle,⁵ in practice they do not appear to search for, understand, trust, or make use of the data. This must have come as something of a shock to the proponents of report cards who expected consumers to respond to the information in a rational way, weighing up the costs and benefits, making a judgement about which providers were best, and driving low quality providers out of the competitive market.

Researchers and policy makers have attempted to explain this apparent paradox in terms of deficiencies in the content, presentation, or dissemination of the data or in terms of the lack of real choice for many US citizens. In doing so they may have missed the point. The apparent disconnection between the public's demand for information and their use of report cards is perhaps not as paradoxical as it first appears. It is possible that service uses are simply saying that they want the information to be available and that they are dissatisfied with what they perceive as the veil of secrecy and professional protectionism currently seen in health care. It is, of course, possible that better data or well informed and empowered consumers will be more willing to use report cards in the future, but it is probable that the impact of report cards on consumer behaviour will always be marginal. Perhaps the rational choice model of decision making, so admired by economists, is an inappropriate way of explaining the public's choice of healthcare providers. Is it possible that the public simply does not want to behave in a consumerist way in all aspects of modern life? Sociobehavioural models of decision making⁶ which recognise the complex input of beliefs, experiences, enabling factors, and the unique self-perception of problems are more useful in explaining the public response to report cards. In an attempt to understand the role of report cards it might then be more productive to re-focus attention on the mechanisms of lay decision making, the merits of expert held knowledge, and the role of advocates in making use of complex comparative information.

Those who are disappointed by the apparent disinterest shown by consumers may gain solace from the second paper on public disclosure in this issue. Davies suggests that US hospitals do make use of comparative information and that the public release of the data acts as a catalyst by reminding, refocusing, or shaming the organisations into giving priority to quality improvement.⁷ Again, the author suggests that the data are not used in an entirely rational way. Respondents tended to use the report cards to confirm their views about the performance of their own and other organisations, views which were based primarily on informal contacts and personal experiences. If the “hard” data did not support their prejudice, then they were more likely to judge the data to be incorrect than to accept that their own views might be wrong.

Nevertheless, the report cards served an important purpose by stimulating the organisations to look beyond the published data and encouraging them to develop and improve their own internal data systems. This suggests that the data contained in report cards have to be accurate enough to engage the attention of those whose responsibility it is to take action, but does not have to be perfect. If correct, this has important policy implications. The New York Cardiac Surgery Report System, for example, disseminated sophisticated risk adjusted data which enabled reasonably valid and reliable judgements to be drawn about the relative performance of individual cardiac surgeons and hospitals in the state of New York.⁸ Such data are extremely costly and time consuming to produce. This might be a necessary expense if the aim of the data is to make proscriptive judgements about fitness to practice, but may not be required if the aim of the report cards is to encourage engagement with the quality improvement process.

The introduction of report cards in the USA is not a shining example of implementing a radical and innovative health policy. The enthusiasm for the public reporting of performance is understandable; it must be right to provide information in an open and democratic society and it must be better for all stakeholders to be informed than to be kept in the dark. However, the initial expectations of report cards in the USA seem in retrospect to be naïve. There can be little doubt that comparative information about quality of care will be freely available in most developed countries within the next decade. Those who are responsible for introducing report cards and those who wish to make use of them would do well to examine the rapidly expanding literature in this field.