CommentaryPublic disclosure of performance data: does the public get what the public wants?
References (11)
- R Smith
All changed, changed utterly
BMJ
(1998) - HTO Davies et al.
Interpreting health outcomes 5DW
J Eval Clin Practice
(1997) - S Edgman-Levitan et al.
What information do consumers want and need?
Health Affairs
(1996) - JH Hibbard et al.
Choosing a health plan: do large employers use the data?
Health Affairs
(1997) - JH Hibbard et al.
Will quality report cards help consumers?
Health Affairs
(1997)
Cited by (44)
Use of benchmarking and public reporting for infection control in four high-income countries
2011, The Lancet Infectious DiseasesCitation Excerpt :In line with the fundamental effect of health-care associated infections (HCAIs) on patients' safety,5,6 HCAI indicators—such as infection rates—have been used widely as benchmarks to enhance quality of care. Public reporting, although focused initially on surgical complications and mortality,7 has been applied increasingly to HCAIs in recent years. Because of a growing demand for transparency and accountability, chiefly by the media, patient advocacy groups, legislative bodies, and accreditation organisations, public reporting of HCAI indicators has been mandated in several countries, including England, France, and the USA.8–10
Benchmarking of surveillance data for health-care-associated infection (HCAI) has been used for more than three decades to inform prevention strategies and improve patients' safety. In recent years, public reporting of HCAI indicators has been mandated in several countries because of an increasing demand for transparency, although many methodological issues surrounding benchmarking remain unresolved and are highly debated. In this Review, we describe developments in benchmarking and public reporting of HCAI indicators in England, France, Germany, and the USA. Although benchmarking networks in these countries are derived from a common model and use similar methods, approaches to public reporting have been more diverse. The USA and England have predominantly focused on reporting of infection rates, whereas France has put emphasis on process and structure indicators. In Germany, HCAI indicators of individual institutions are treated confidentially and are not disseminated publicly. Although evidence for a direct effect of public reporting of indicators alone on incidence of HCAIs is weak at present, it has been associated with substantial organisational change. An opportunity now exists to learn from the different strategies that have been adopted.
Consumer use of publicly released hospital performance information: Assessment of the National Hospital Evaluation Program in Korea
2009, Health PolicyTo assess the extent of consumer use of publicly released hospital performance information by the National Health Evaluation Program (HEP) in Korea.
A questionnaire survey with 385 outpatients visiting four general hospitals in Seoul.
The consumer use of performance information was assessed by the consumers’ intention to: (1) recommend hospitals with good performance reports, according to HEP, to their relatives; (2) switch to other hospitals with a better performance and (3) keep the performance report for future use.
Overall, 52–75% of the respondents expressed their intention to use the hospital performance information. Logistic regression analysis results showed that people would use the performance information if they considered HEP to be effective in improving the quality of health care and the performance reports to be trustworthy and useful in choosing hospitals.
This study provides evidence that consumers in a health care system with few restrictions for provider choice, such as in Korea, have a high potential to utilize the provider performance information in their decision making. If public use of the performance information becomes common, policy makers should acknowledge the critical value of the quality of the performance report in order to avoid misleading consumers.
Validation of the EuroSCORE probabilistic model in patients undergoing coronary bypass grafting
2008, Revista Espanola de CardiologiaEuroSCORE es un modelo probabilístico para estimar la probabilidad de mortalidad hospitalaria en pacientes sometidos a cirugía cardiaca. Es un instrumento útil para evaluar la calidad asistencial. Existen dos variantes del modelo, el EuroSCORE logístico (EU-L) y el EuroSCORE aditivo (EU-A). El objetivo del estudio es validar el modelo EuroSCORE en pacientes intervenidos en el Hospital Clínic de Barcelona y comparar los resultados de las dos variantes del modelo.
Se ha incluido a los pacientes intervenidos de injerto coronario en el Hospital Clínic de Barcelona durante 2 años consecutivos. Se ha validado el modelo a partir de su capacidad de calibración (prueba de Hosmer- Lemeshow) y discriminación (área bajo la curva ROC). Se han comparado los dos modelos con un análisis descriptivo de la media de la mortalidad para el total y según grupos de riesgo y mediante su poder de discriminación.
Un total de 498 pacientes fueron intervenidos e incluidos en el estudio. La calibración del modelo es satisfactoria (p = 0,32) y el área bajo la curva ROC es de 0,83. La mortalidad hospitalaria observada alcanzó el 5,8% y la estimada, el 4,2% (EU-L) y el 3,9% (EU-A). Se han observado mayores diferencias en el grupo de pacientes de alto riesgo, en los que la mortalidad predicha por la variante logística se aproxima más a la mortalidad real.
EuroSCORE ha sido validado adecuadamente y puede utilizarse para medir los resultados de la práctica asistencial. El modelo logístico se aproxima más a la mortalidad real en el grupo de pacientes de alto riesgo.
EuroSCORE utilizes a probabilistic model for predicting the risk of in-hospital mortality in patients undergoing cardiac surgery. It is a useful instrument for evaluating quality of care. The model has two variants: the logistic EuroSCORE and the additive EuroSCORE. The aim of this study was to validate the EuroSCORE model in patients undergoing surgery at Hospital Clínic in Barcelona, Spain, and to compare the results obtained with the two variants.
The study included all patients who received a coronary artery bypass graft (CABG) at Hospital Clínic in Barcelona in two consecutive years. The model's validity was assessed on the basis of its calibration (using the Hosmer-Lemeshow test) and its discrimination (using the receiver operating characteristic [ROC] curve). The two models were compared by carrying out a descriptive analysis of mortality for the whole group and for different risk groups, and by determining the models’ discriminative power.
A total of 498 patients underwent CABG surgery and were included in the study. The Hosmer- Lemeshow test showed that the model's calibration was satisfactory (P=.32) and the area under the ROC curve was 0.83. The observed in-hospital mortality rate was 5.8%. The predicted rate was 4.2% with the logistic EuroSCORE and 3.9% with the additive EuroSCORE. Large differences were observed in high-risk patients. In these patients, the mortality predicted by the logistic variant was closer to the actual mortality.
EuroSCORE's validity was found to be satisfactory and the model can be used to evaluate quality of care. In high-risk patients, mortality estimated using the logistic model was closer to the actual mortality.
Analysis of Spanish hospital's reputation: Relationship with their scientific production in different subspecialities
2006, Medicina ClinicaEvaluar la reputación de los hospitales y analizar su posible relación con la producción científica (artículos y citaciones) de dichos hospitales.
Encuesta a miembros de las Sociedades Españolas de Cardiología, Patología Digestiva, Neurología y Otorrinolaringología. Cada socio podía votar 5 hospitales que consideraba de mayor reputación asistencial en su especialidad. La puntuación final de los 15 hospitales con más del 5% de los votos constituye el índice de reputación. Se evaluó la producción científica (publicaciones y citaciones) en cada especialidad y para cada hospital más votado y se analizó la posible relación entre ambos parámetros.
Se obtuvieron 151 (30%) respuestas en cardiología, 227 (27%) en patología digestiva,148 (30%) en neurología y 177 (18%)en otorrinolaringología. Los índices de publicaciones y de citaciones mostraron mayoritariamente una relación positiva y estadísticamente significativa con el índice de reputación.
El trabajo presentado realiza una primera aproximación al estudio de la reputación como medida indirecta de la calidad de los hospitales españoles. La producción científica de éstos mostró una relación positiva con el índice de reputación.
To assess the relationship between reputation of hospitals, as determined by specialists’ opinion, and their scientific production.
A questionnaire was sent to a sample of members of the Spanish Societies of Cardiology, Digestive Diseases, Neurology and Otorhinolaryngology. Each member had the possibility to nominate 5 hospitals with the greatest reputation amongst his/her specialty. Final score for those hospitals with more than 5% of all votes was used for the development of a reputation index. We assess the scientific production (number of publications and citations) for nominated hospitals and analyze the possible correlation between the reputation index and the scientific production.
We received 151 (30%) answers for cardiology, 227 (27%) for digestive diseases,148 (30%) for neurology and 177 (18%) for otorhinolaryngology. The number of published articles and the number of citations for each nominated hospital showed a positive and statistically significant correlation with the reputation index.
This is the first approach to analyze reputation parameters for Spanish hospitals. Scientific production and reputation index showed a positive correlation.
Theory and practice of waiting time data as a performance indicator in health care: A case study from The Netherlands
2005, Health PolicyIn this article we investigate the use of waiting time data as a performance indicator in health care in The Netherlands. We explain why the current publication of waiting time data fails to achieve one of the main goals: to have consumers and general practitioners act upon this information. The reason, we claim, is that even seemingly clear-cut, easily measurable and objective numbers such as waiting times need interpretation to become meaningful. Discussing four themes – the patient behind the number, the treatment behind the number, the strategy behind the number, and the specificity of the number – we will discuss just how deeply this need for interpretation affects the usability of ‘waiting times’ for purposes such as informing consumers. We will argue that this problem is due to not making a clear distinction between performance indicators for internal use and for external use. We conclude that the usefulness of the publication of waiting time data for consumers strongly increases when waiting times are guaranteed and related to treatment options like booking possibilities and other performance indicators such as patient satisfaction.
Consensus on a process of benchmarking in primary care in Barcelona
2005, Atencion PrimariaDefinir la estrategia, el marco conceptual, la metodología y los indicadores que han de impulsar y consolidar la cultura de la referencia externa (benchmarking) como estrategia de cambio en los equipos de atención primaria (EAP).
Estudio descriptivo, transversal.
Servicios de atención primaria (AP) de la Región Sanitaria de la ciudad de Barcelona.
Se distinguen 2 fases. En la primera se constituyó un grupo asesor (GA) que se organizó en 4 grupos focales, donde se acordaron los ejes, el marco conceptual, las dimensiones, los indicadores y la metodología para la comparación de los EAP. La segunda fase, de priorización, se realizó a través de una encuesta a líderes de opinión, quienes para cada uno de los indicadores propuestos valoraron el grado de acuerdo y la adecuación de los indicadores, su relevancia, la capacidad de la AP para modificar el resultado y la factibilidad de la información para su elaboración.
La implicación de los profesionales, la orientación hacia la mejora, la transparencia y la difusión de la evaluación fueron identificados como elementos estratégicos de la dinámica de benchmarking. De acuerdo con los principios básicos de la AP y del sistema sanitario, se concretaron 6 dimensiones de evaluación: accesibilidad, efectividad, capacidad resolutiva, longitudinalidad, relación costeeficiencia y resultados. En 43 de los 57 indicadores priorizados se obtuvo el consenso de más del 90% de los consultores.
La evaluación como instrumento útil para la gestión de la calidad de la AP ha de generar acciones de mejora o cambio en los EAP. La implicación de los profesionales en su diseño y desarrollo puede ayudar a su aceptabilidad y a la implementación de las acciones de cambio que se deriven. Es necesario evaluar los indicadores utilizados y analizar la influencia que ha tenido la política de benchmarking en los resultados de la prestación de los servicios de AP.
To define the strategy, the conceptual framework, the methodology and the indicators that are needed to promote and consolidate the culture of external reference (benchmarking) as a strategy for change in Primary Care teams (PCT).
Cross-sectional, descriptive study.
Primary care services of the Barcelona City Health Region.
Two stages were distinguished. At the first stage, an adviser group was set up. This was divided into 4 focus groups in which the main lines, the conceptual framework, the sizes, the indicators and the methodology for comparing PCTs were agreed. The second stage, that of prioritisation, was conducted by means of a questionnaire to opinion-formers. For each of the indicators proposed, they appraised the degree of agreement, the suitability and relevance of indicators, the capacity of PC to modify results and the practicality of the information for composing the indicators.
The involvement of professionals, their approach to improvement, and the transparency and dissemination of the evaluation were identified as strategic elements of benchmarking dynamics. In line with the basic principles of PC and the health system, 6 dimensions for evaluation were set: accessibility, effectiveness, capacity to resolve problems, longitudinality, cost-efficiency, and results. 43 of the 57 indicators prioritised gained the consensus of over 90% of the consultants.
Evaluation as a useful tool for managing PC quality has to generate improvements or changes in PCTs. The involvement of professionals in the design and development of evaluation may help both its acceptance and the implementation of the changes arising from it. The indicators used and the effect of benchmarking policy on the results of PC service delivery require evaluation.