Background

Since the first publication of hospital mortality rates in 1986 by the federal agency that administers the US Medicare insurance programme, public disclosure of information about the quality of health plans, hospitals, and doctors has provoked controversy.¹ Publicly disclosed performance reports, sometimes called “report cards”, are one manifestation of a health care marketplace in which competing providers would measure and report information about the quality of care they offer.² According to the proponents of market theory, US employers, who pay a major share of health care costs in the USA, would purchase prepaid comprehensive care through health plans rather than reimbursing providers for each service.³ Employers would use report cards to choose high quality health plans at a reasonable cost. Insured employees would select health plans, hospitals, and doctors using report card information about providers' quality, price, and accessibility. As a result, health plans, hospitals, and doctors with the best performance records would gain market share by improving the quality of services and becoming more efficient.⁴ The allure of a market based on “consumer choice” is such that other nations are now trying to emulate its principles as they reform their own health systems.⁵

Ironically, the USA has not itself realised this ideal “consumer choice” market. Performance measures have not moved the marketplace and consumers have not flocked to better performing health plans, hospitals, or physicians. Instead, while quality and performance results have not dramatically influenced the choices of individual or group purchasers, they appear to have stimulated some improvements in health care, probably by revealing poor quality practices to organisational managers. The actions of these managers may have improved some elements of clinical care for those enrolled in managed care between 1996 and 1999. For example, the number of children receiving varicella immunisation has increased from 40% to 64%, the percentage of heart attack victims receiving beta-blocker medication increased from 62% to 85%, and the percentage of adolescents receiving measles, mumps, and rubella immunisation has increased from 52% to 59%.⁶

In the past decade, coalitions of large corporate employers using their purchasing leverage have stimulated an expansion in the number of programmes that measure and publicly disclose information about quality.⁷ In 1992 Consumer Reports was the first organisation to rate health plans by asking a large number of subscribers about their experiences in managed care. Subscribers rated overall satisfaction as well as a variety of dimensions of service quality such as waiting times for non-emergency appointments, waiting times on the telephone, and in doctors' surgeries. At the same time a non-profit organisation, the National Committee for Quality Assurance (NCQA), began to accredit health plans and to assess their performance using a standardised set of measures known as HEDIS^®, the Health Plan Employer Data and Information Set.

HEDIS was designed primarily to gauge the underuse of medical services in managed care plans. In developing and expanding HEDIS over the years the NCQA has convened representatives of employers, health plans, and consumers to guide policy on performance measurement. According to the NCQA model, employers would offer their employees choice among only the health plans with the best performance records. In choosing from preselected health plans, employees would use report cards to discriminate further. The federal government also contributed by financing development of the Consumer Assessment of Health Plans (CAHPS^®), a standardised survey of health plan members that provides data for plan comparisons. The government also required the health plans serving Medicare beneficiaries with which it had a contract to report HEDIS and CAHPS data.

This proliferation of programmes has spawned concerns about adverse effects of publicly disclosing information about health care quality. Some observers have worried about the potential for performance results to mislead patients, because the measures address only a small fraction of the care provided by doctors and may not be sufficiently standardised or adjusted for important differences in the health risks of providers' populations.^8–12 Also, providers could ignore unmeasured aspects of practice and concentrate on measures that show them in a favourable light or “game” measurement systems by refusing to care for the most ill patients or by recording inaccurate data.¹³ The expense of generating rigorous performance reports might divert resources from patient care.¹⁴ Others have worried that two key audiences—purchasers and patients—would be unable to use this novel information to aid in the decision making and choice of providers.^15–18 Providers might become indifferent or hostile to performance measurement. For example, NCQA has faced considerable difficulty convincing employers that there is a business case for improving quality through report cards. “Report cards don't translate into a market advantage”, says Allan Greenberg, former CEO of Harvard Pilgrim Health Care. “The small and mid-size employers are not involved in that movement. The larger ones do care, but will they pay more for good report cards? No.”¹⁹

In this contentious environment US policy makers have gained considerable knowledge about the effects of a public disclosure strategy.²⁰ This review describes the lessons of the public disclosure movement in the USA and focuses on the effects of disclosure on the processes and outcomes of health care, the effects on choices by individual and group purchasers, the effects on health plans' selection of providers, and the response of the media to demands for information about quality. We then speculate about the value of public disclosure and the future directions that it might take.

Evidence for the effectiveness of public disclosure

Much of what we know about the effects of public disclosure of information about health care quality is based on experience with four programmes: (1) disclosure of risk adjusted hospital mortality rates, (2) disclosure of risk adjusted mortality rates after coronary artery bypass graft surgery, (3) disclosure of health plan performance using HEDIS measures of the quality of preventive care, and (4) disclosure of patient and enrollee experiences with the quality of health plan and hospital care using a variety of surveys.

To evaluate the impact of public disclosure, researchers have looked for evidence of direct and indirect effects of the information. Drawing on market theory, some researchers have hypothesised that, in the wake of disclosure, individuals choose higher quality providers leading to measurable improvements in population health outcomes. They also suggest that, for this to occur, a series of measurable mediating events are necessary: purchasers and consumers should have access to report cards, obtain them, interpret them correctly, and switch providers or health plans, appropriately increasing the market shares of high quality providers. Other researchers have postulated that providers will use performance reports to improve practices even if consumers do not use them to make decisions. We review existing research, explore some of the institutional responses, and describe the influence of the media on the demand for publicly disclosed information about quality over the past decade.

Effect on processes and outcomes of care

A handful of studies have attempted to evaluate the impact of public disclosure programmes on the processes and outcomes of care. The NCQA has found that health plans have increased the delivery of preventive services that were the subject of measurement and public disclosure. In general, observational studies of outcomes have compared mortality rates and other clinical “outcomes” (caesarean section rates and vaginal birth after caesarean rates) before and after statewide or local public disclosure programmes began.^21–25 All these studies found small improvements in measured outcomes; however, none involved a control group that did not receive public information, so the relationship between improved outcomes and public disclosure remains speculative. For instance, rates of mortality for coronary artery bypass graft surgery appeared to be declining over time at similar rates in neighbouring states that did not have public disclosure programmes, suggesting that improvements by providers, rather than selection of high quality surgeons by consumers, was more likely to explain the decline.²⁶²⁷

There are many reasons why processes and outcomes might be slow to change in response to new information. Barriers such as lack of awareness of the information, a lack of salience or interpretability, or the inability to switch providers because of constraints related to insurance or travel distance could interfere with the use of information about quality. Purchasers might have existing contracts with selected providers that cannot be altered immediately. Poorly performing hospitals might be able to maintain the flow of patients by reducing their prices and thereby retaining contracts with purchasers. For the last 6 years, New York State has published risk adjusted data on mortality rates after cardiac surgery. In two studies, managed care organisations did not appear to contract with hospitals that had lower mortality rates. Indeed, they may have been more likely to contract with hospitals that had higher mortality rates, presumably because of pre-existing contracts or because they were able to obtain lower prices.²⁸²⁹

Barriers to consumer use of information

In light of the difficulty of interpreting broader changes in processes or outcomes of care, many studies have examined the direct effects of disclosing information to purchasers and consumers. A growing body of literature suggests that neither purchasers nor consumers make use of publicly disclosed information about quality.²⁰ This failure appears to be related to the shortcomings of report cards, the complexity of the task for individual consumers of obtaining and comparing information and, perhaps most important, the limited salience of the information.¹⁷

Much time, effort, and money has been expended to identify the best formats for disclosing information about quality—for instance, whether numbers, stars, bars, circles, or graphs are most informative. Despite this, most report cards are not useful documents. Consumer Reports, the well known independent magazine that tests and rates products and services, examined 30 report cards intended to help employees and Medicare and Medicaid recipients choose a health plan. All failed to tell consumers how to use the data or how to make trade offs. Most did not present price information that would help users to make a trade off between the premium required and the quality of services. Many report cards overwhelmed consumers with too much information and offered explanations that were confusing or misleading. Sometimes unrelated dimensions were mixed up and rolled into a single measure that obscured the significance of each. The report cards did not rank plans or make judgements, but left it to consumers to figure out how to interpret the data. For political reasons, sponsors of report cards have hesitated to offer such judgements.¹⁹

Efforts to enhance presentation formats may well be a smokescreen that obscures a more fundamental problem—namely, the relative weakness of quantitative data compared with personal past experience, recommendations from trusted professionals, or recommendations from informal networks of family and friends. Consumers have not perceived report card information to be useful because the indicators are not salient. They do not view all quality indicators as equally useful, and they frequently do not share the assumptions that underpin quality indicators.³⁰ To date, the developers of HEDIS and CAHPS have considered their measures to be important proxies for the overall quality of health plans—an assumption that remains untested. No published studies have assessed whether or not health plans with higher HEDIS or CAHPS scores also have better clinical outcomes or provide more appropriate medical care.

It is important to note that HEDIS assesses underuse, but not overuse or misuse, which are equally important dimensions of quality. Neither HEDIS nor CAHPS measures the appropriateness of care (whether patients will receive the right diagnosis and the right treatment at the right time). The rate of mammography in a health plan does not tell patients whether they will get appropriate follow up treatment for breast cancer. Nor do HEDIS or CAHPS tell patients facing potential back surgery whether the doctor accurately determined their need for the operation or whether the back surgeon has performed the procedure often enough to qualify as competent. Misuse and overuse are highly salient to patients but are still unmeasured.

In the absence of salience, quality indicators do not outweigh other factors such as price or the recommendations of family and friends. In a controlled experiment in which subjects were paid to review simulated quality information on health plans and then choose from among them, consumers were willing to choose low price, high quality health plans, but tended to be more influenced by the benefits available and the price. A large percentage of subjects were willing to keep a more expensive health plan with more benefits even if it had lower quality ratings.³¹

Assuming the availability of valid, reliable, and credible report cards, there are many other reasons why consumers might fail to use them to drive competition and improve quality. Firstly, consumers must believe that quality varies. If they do not, then they will not seek information to learn about differences in quality. Secondly, consumers must believe that they have a choice of health plans, doctors, and hospitals. They must have timely access to information about quality for real time decision making, they must trust the source of the information, and they must be able to interpret it correctly—that is, they must understand the information sufficiently to be able to make a better choice.

Failure to satisfy even one of these preconditions can render a report card useless as a decision tool. For example, in a national survey fewer than 50% of Americans believed that there are “big differences in quality” among health care providers such as health plans, hospitals, and primary care providers.¹⁸ Less than one third believed that there are “big differences in quality” among specialists. This suggests that about half of all Americans would not even seek information about quality differences. Another study reported that only a minority of Americans have a choice among health plans.³² Those without a choice would be unlikely to make use of health plan report cards. Less than one third of Americans reported that they had seen any information on health care quality. Even though the vast majority believed the information would be useful, very few reported using it.¹⁸ A related series of studies in which clinical investigators carefully interviewed individuals about the content of report cards revealed that, in the context of health care, consumers did not easily understand information about quality and even had problems interpreting price information.³⁰

Failure of highly salient information to reach patients

Even under the best conditions when information is highly salient, it is not clear that consumers use it. In Pennsylvania, a state with a public disclosure programme that lists hospital specific and surgeon specific risk adjusted mortality rates, cardiac surgery patients who had recently undergone coronary artery bypass graft surgery did not obtain or use the information. Fewer than 20% of the patients were aware of the mortality rates, fewer than 12% reported that they saw the information before having an operation, and fewer than 1% reported that the information had influenced their choice of hospital or surgeon.³³ The state agency had engaged in extensive dissemination efforts that included newspaper stories and public service announcements in the media. In addition, it made the reports available free by mail as well as on a web site (www.phc4.org). In general, most patients accepted information about quality from their referring cardiologist. A related study found that cardiologists rarely informed patients of the existence of the risk adjusted operative mortality rates.¹³

The situation was similar for information on the quality of long term care facilities (nursing homes). Among readers of Consumer Reports who had recent experience with long term care, only 6% had used the results of comprehensive state surveys detailing lapses in quality of nursing homes. The reports are published to assist people in choosing a long term care facility.³⁴

Can information on quality reshape public opinion?

By shaping the opinions of relatives, friends, or trusted clinician advisors, publicly disclosed quality information could conceivably alter the choices patients make without their knowledge.¹⁸³⁵ This effect, which would drive consumers to “vote with their feet”, should be measurable as a rise in market share of hospitals or health plans identified as high quality performers and a shrinking of market share among those identified as low quality performers.

Few studies have examined the possibility that market shares change in the wake of public disclosure. After the Health Care Financing Administration (HCFA) published risk adjusted hospital mortality data in 1986, one study found no change in occupancy rates among hospitals.³⁶ As in Pennsylvania, New York's cardiac surgery reporting programme disseminates results annually, publishing them in major newspapers and posting them on its web site (www.health.state.ny.us). Even though risk adjusted mortality rates after cardiac surgery declined rapidly in New York State once the public reporting programme began, surgical volumes did not rise or fall significantly at hospitals with high and low mortality rates.²⁵ Another study of the New York programme suggested that surgeons with lower mortality after cardiac surgery might have increased their personal market share of surgical operations after public disclosure began, but the reasons for this growth remained uncertain.³⁷

Role of corporate purchasers in selecting high quality providers

Given the challenges faced by individual consumers, some researchers have advocated that purchasers interpret information on quality and guide consumers towards higher quality health plans and hospitals using contracting leverage or financial incentives.³⁸ This approach, sometimes labelled “value based” purchasing, includes selective contracting with high quality, low cost providers, availability of comparative performance reports (or “report cards”) at the time of “open enrolment” when employees are selecting insurance plans, and performance based premium sharing in which the employer subsidises a greater percentage of the insurance premium for high quality health plans, making these arrangements less costly to the employee than lower quality health plans.³⁹

Research suggests, however, that few purchasers or employer coalitions use information about quality in a meaningful way. Surveys of employers in 1996 and 1997 showed that only a third were aware of a health plan accreditation programme offered by the NCQA. Only 9% of employers insisted that the health plan should pass accreditation, and only 1% provided HEDIS data to employees.⁴⁰ Only 15% of these firms thought it was “very important” to provide HEDIS data to employees. Larger employers have been the main proponents of health care performance measurement and public disclosure. They were more likely than smaller employers to use performance standards in contracting decisions with health plans.

Health plans with poorer scores on various dimensions of quality have not withered away. Contrary to conventional market theory, some plans with the worst scores for consumer satisfaction have indeed become dominant in the marketplace. Cigna, Humana, and US Healthcare (now Aetna US Healthcare) have been ranked towards the bottom in Consumer Reports rankings in 1992, 1996, and 1999, and we have no reason to believe there will be a change in the next report due in the autumn of 2001.¹⁹⁴¹⁴² One reason for the lower satisfaction scores for US Healthcare may rest with the plan's tighter control of medical services. This enabled the plan to control costs and pass on the savings to employers in the form of lower premiums which is what employers demanded (Dr Carol Diamond, former President of US Quality Algorithms, personal communication). Furthermore, many health plans thrive even though they have refused to allow public disclosure of their HEDIS performance. For many years the Cigna health plan did not allow the NCQA to release its performance data to the public. In fact, 21 out of 54 health plans rated by Consumer Reports in 1999 refused to allow disclosure of their HEDIS performance by NCQA.¹⁹ These 21 plans scored significantly lower than the other plans in the Consumer Reports study. In addition, health plans that reported HEDIS indicators to the NCQA, but refused to allow the NCQA to disclose them publicly, had lower scores on the HEDIS indicators than other health plans.⁴³

Cigna, Humana, and what is now Aetna US Healthcare are among the most successful major national health plans, suggesting that factors other than patient satisfaction influence an employer's decision to offer particular health plans. Prompt payment of claims, large networks, and low prices are paramount in employers' decisions.⁴⁰ At the same time, US Healthcare was an innovator in quality improvement programmes but, according to Diamond, the marketplace did not reward the plan for those efforts. High ratings and high satisfaction do not translate into increased enrolment or financial success. Harvard Pilgrim Health Care exhibited both high HEDIS scores⁴³ and excellent satisfaction scores,¹⁹⁴² but in early 2000 the plan had plunged into financial difficulty.

In the USA, smaller employers (with fewer than 50 employees) represent 96% of the job market. Among these employers, quality of care and accreditation ranked fifth and sixth out of six factors considered in buying health insurance coverage for their employees. Price, benefits packages, and physician availability ranked higher.⁴⁴ Some employers have pooled their buying power by forming or joining purchasing coalitions.⁴⁵ Whether these coalitions use information about quality is not clear. Among employers that were members of health care purchasing coalitions, most stated that they were responsible for the quality of health plans offered to employees, but the primary factors considered were the geographical coverage offered by health plans and access of members to services.⁴⁶

Some purchasing coalitions have been effective in negotiating price discounts with health plans, in disseminating information about quality (primarily HEDIS data and satisfaction surveys), and in improving quality.⁴⁷ In particular, the Pacific Business Group on Health incorporated publicly disclosed HEDIS information into contract negotiations and put more than $8 million in premium payments in a risk pool to be paid only to health plans that reached performance targets. It appeared that HEDIS data were a valuable part of the process of improving care.⁴⁸ In a highly structured regional purchasing arrangement through which a purchasing coalition buys health care services from care systems with non-overlapping providers (as opposed to health plans that may include many of the same providers in each of their networks), the Buyer's Health Care Action Group (BHCAG) demonstrated that care systems with higher publicly disclosed satisfaction scores appeared to be gaining enrolment compared with those with lower satisfaction scores.⁴⁹⁵⁰

Few studies have examined whether or not performance reports provided by employers have had an impact on employee selection of health plans. In one large company, employees were less likely to select health plans with high satisfaction ratings but were more likely to select health plans with high preventive service scores.⁵¹ Other aspects of health plan performance had no relationship to the choice of health plan, but the higher the premium, the lower the probability that employees would select a particular plan. In a related study there appeared to be no relationship between the publicly disclosed HEDIS scores of a health plan and its share of enrolment when controlling for other factors, which suggests that the information did not encourage enrolment in high quality health plans.⁵²

It has also been suggested that health plans might use information about quality on behalf of their members to select specialists and hospitals for their networks. However, two recent studies suggest that health plans did not use risk adjusted mortality data to obtain cardiac surgery services at hospitals with lower mortality rates and, instead, may have contracted selectively with hospitals that had higher mortality rates.²⁸²⁹ Some hospitals have demanded “non-steerage” clauses in health plan contracts to prevent selective contracting.

The media and demand for public disclosure of information on quality of health care

By 1996 when Consumer Reports surveyed subscribers about their health plans for the second time, new raters had entered the field. The availability of HEDIS scores from the NCQA, as well as data on satisfaction, enabled US News and World Report and Newsweek magazines to compile their own ratings of health plans. In the mid 1990s news organisations rushed to publicise the initial release of HEDIS data. In many instances they made inferences which may have gone beyond what the data would support. In 1996 Consumer Reports noted that the numbers from HEDIS data were “insufficient to measure quality”. US News published annual ratings up to 1999 and then stopped because the chief editor believed that consumers in most areas had no choice (Avery Comarow, Editor, America's Best Hospitals, personal communication). Newsweek continues to publish ratings to coincide with the annual release of NCQA data. The NCQA publishes information about quality as part of a product called Quality Compass, of which an interactive version is available on its website (www.ncqa.org). The current version of Quality Compass combines the results of the NCQA accreditation surveys with HEDIS measures. The results are presented in five groups: access and service, qualified providers, staying healthy, getting better, and living with illness. Many dimensions are rolled into one single composite represented by stars. Underlying data are not readily available for consumers who wish to probe behind the combined scores. The release of the NCQA data generates a lot of press attention from local newspapers and television stations which publicise the relevant results for their geographical areas. Some states such as Maryland, New Jersey, New York, and Pennsylvania also make available report cards giving HEDIS scores for health plans operating in their states. Newspapers sometimes publish information from these report cards. The effects of these efforts remain largely unstudied.

US News continues to publish an annual ranking of hospitals. Using data such as mortality rates, ratings provided by top specialists, and data from surveys conducted by the American Hospital Association, the report lists the 173 best hospitals in the country.⁵³ Although one study confirmed that highly ranked hospitals were more likely to provide cardiac care consistent with guidelines for patients with myocardial infarction, there has been little or no evidence that patients are selecting hospitals according to the US News ranking. Patients most often go to the hospital recommended by their doctor, and location may be the most important factor used in selecting a hospital. Even when health plans have tried to steer members to particular hospitals that were centres of excellence for a particular procedure or treatment, patients have refused to go, arguing that health plans are sending them to these hospitals to save money (Dr Andrew Weisenthal, Kaiser Permanente, personal communication).

US News has no quantitative data on how or whether its rankings are used. According to Avery Comarow, Editor of America's Best Hospitals, US News receives twice as many calls from hospitals and health systems as it does from consumers. Hospitals and health systems want to learn what they can do to be listed. “It appears that hospitals will go to great lengths to appear on these lists,” says Comarow, “What started out as something to bring service to consumers has turned into a marketing tool.” The media have played a leading role in promoting the use of public information on quality, but it is not clear that consumers are demanding such information.

Resistance of providers to performance reports

During the past few years providers have increased their resistance to further attempts to make public the data on quality. For example, Consumer Reports has had difficulty in getting health plans to provide data to augment survey results from its members. Those that do participate often give skimpy answers that appear as if public relations officials have prepared them or they delay participation until the data are no longer relevant. Some providers are simply reluctant to participate in voluntary efforts to improve quality. The California Healthcare Foundation and the California Institute for Health System Performance, an affiliate of the California Healthcare Association, are sponsoring a project called “Patient Evaluation and Performance in California Hospitals.” Some 300 hospitals have been asked to field a survey to patients inquiring about their hospital stay. The data from a standardised instrument are to be made public later this year. Only 100 hospitals representing 50% of the beds have participated (Ann Monroe, Director, Quality Initiative, California Healthcare Foundation, personal communication).

Conclusions and future directions

The US experience with public disclosure is in many ways paradoxical. Consumers express a desire for more information about quality while, at the same time, few are able to understand or use it. Producers of report cards continue to refine measures and create new presentation formats for patients and purchasers even though there is no evidence that these groups choose health plans or providers based on report card information. Despite the fact that consumers and purchasers do not use the information to guide their selection of providers, outcomes and performance appear to be improving in some areas. If public disclosure of report cards has an effect, it is because disclosure focuses health plans, hospitals, and other providers to address issues of health care quality that may in the past have been ignored.^2454–56 By standardising the measurement of quality, HEDIS has created expectations about performance and helped organisations to identify gaps in care. This may be its greatest contribution. Provider organisations have made efforts to measure and increase performance rates by using measures for internal monitoring and to set financial rewards and penalties for providers.⁵⁷⁵⁸ Report cards on rates of mortality after cardiac surgery have directed hospital managers and surgeons to problem areas, forcing them to examine the process of surgical care and to improve it. Public disclosure appears to galvanise providers into action, either because they perceive a threat to their reputations or because it is an opportunity to market their excellence.²⁰²⁷

Will the “sleeping giant” of consumer choice awaken if we discover the right quality measures, the right report formats, the right dissemination strategy, or the right methods for educating consumers to use report cards? Would measures that evaluate individual physicians on a variety of performance dimensions be more useful to consumers? We do not think so. Evidence suggests that voluntary reporting by health plans and hospitals is inadequate, that purchasers are losing interest, that costs are too high, and provider resistance too formidable.

Except in the case of mandates from public insurance programmes, health plans and hospitals have not been required by law or regulation to report performance data for public disclosure. While many health plans and hospitals do report data voluntarily or at the request of corporate purchasers, in 1996 65% of health plans did not participate in the NCQA's HEDIS reporting and resistance to reporting may be growing.⁵⁹ In Cleveland, Ohio the Cleveland Health Quality Choice programme, one of the more ambitious and successful pioneering efforts to report hospital quality, ceased operations when a large number of hospitals refused to report performance data.⁶⁰

Some groups such as the Leapfrog group, an organisation of employers, frustrated with their inability to obtain useful data on performance are asking instead for other data on which to rate quality. They have abandoned performance measures in favour of “structural” measures such as the presence of automated drug entry ordering systems in hospitals or intensive care staffing ratios. While such information may be useful to assure a minimum standard of quality in a resource constrained system, this seems to reject the notion of accountability and competition based on performance.

The health care system simply cannot afford the cost of making a quality measurement set sufficiently encyclopaedic to meet the needs of every patient needing care for every condition. We have probably reached the end of efforts to improve presentation formats for information about quality. Even if we perfected collection and formatting, dissemination would continue to be a formidable hurdle. Geography and travel costs constrain consumers' choice among providers for specialised health services. Consumers are unlikely to have the time, money, or inclination to obtain the specialised training that would make them expert in the interpretation of dozens of measures of quality.

Providers have found it easy to undermine the information on quality if it bites them. In Pennsylvania some cardiac specialists persuaded patients that hospitals with low mortality rates following cardiac surgery were not better, but instead were unwilling to operate on the most ill patients who were more likely to die. Reports on nursing home facilities that disclose the results of state inspections are required by law to be posted in every nursing home across the country. This is an excellent source of consumer information that details the quality of care given in a facility. However, a study by Consumer Reports in 1995 found that the reports of these inspections were unavailable, hidden, out of reach, out of date, or unreadable in 37 out of 52 nursing homes.⁶¹

Many have suggested that the internet could make the consumer's task more manageable. In view of the scant use of information on quality that we cite, the proliferation of health web sites that serve consumers may seem paradoxical, but it is not. Consumers actively seek information about diseases that they have and treatments they are considering. This is not a new phenomenon, but has existed since before the early days of organised medicine. The internet provides information primarily about diseases and treatments, but not about performance. The suspect credibility of many web sponsors makes it unclear whether the web can offer unbiased and believable information on the quality of competing provider organisations.

Finally, all efforts to encourage consumers to select providers are aimed at those fortunate Americans who have health insurance. The uninsured endure severe quality problems in health care, and performance measures do not address their problems.⁶² Outside the Medicaid programme, which insures some of the poor and has moved beneficiaries into managed care while offering report cards based on HEDIS and CAHPS measures, there is no targeted dissemination of report cards to the poor or uninsured, nor is it clear how they could be useful in the absence of a range of care options. Community health centres, an important resource for poor and uninsured patients, have been unable to muster the resources for performance measurement.

A sense of urgency about quality of care and the safety of health care systems in the USA is fuelling continued interest in public reporting on quality and has led to proposals to report physicians' error rates.⁶³ New reporting initiatives that extend HEDIS and CAHPS to address the care of patients insured by traditional insurance (rather than managed care) are under way and new uses of the information for public health monitoring are of great interest.^64–68 Medical professional societies such as the American Society for Clinical Oncology and the American Diabetes Association are sponsoring initiatives to develop new quality measures in their areas for treatment and management of chronic disease. It may be that medical professionals are beginning to see the value of performance measurement to guide quality improvement. It is possible that patients may be more interested in measures that assess the care of chronic conditions.

Although consumers may not use publicly disclosed information on quality, we believe that quality measurement should continue and that results should be made available to the interested public. Keeping in mind that performance reports are not the engine that will drive consumers to choose high quality providers, organisations such as the NCQA have an important leadership role and should continue to expand the dialogue among stakeholders, including purchasers, consumers, and providers. These organisations should continue the remarkable voluntary effort to expand and refine measures, to reduce the costs of measurement, to make data collection more efficient, and to understand the use of report cards—not as a guide for consumer choice but as a way to bring about system-wide quality improvement.¹¹

Voluntary efforts will not, however, be adequate to assure widespread quality improvement. Some form of legal mandate may be necessary to ensure that all providers collect performance data and participate in quality improvement. Information about quality must continue to be publicly available. If it is not, there will be no assurance that provider institutions will pay attention to medical care quality. In the USA publicly disclosed information about health care quality can be used to establish a public consensus about the need for quality improvement and the targets for that improvement. Such information will also provide a necessary check on the activities of a largely profit making health care industry in which investors' goals may supersede the focus on quality. But public disclosure of information about the quality of health care is a weak strategy for ensuring quality.