Article Text

PDF
Effect of data validation audit on hospital mortality ranking and pay for performance
  1. Skerdi Haviari1,2,
  2. François Chollet1,
  3. Stéphanie Polazzi1,
  4. Cecile Payet1,
  5. Adrien Beauveil1,
  6. Cyrille Colin1,2,
  7. Antoine Duclos1,2
  1. 1Pôle Information Médicale Evaluation Recherche, Hospices Civils de Lyon, Lyon, France
  2. 2Health Services and Performance Research lab (HESPER EA7425), Université Claude Bernard Lyon 1, Lyon, France
  1. Correspondence to Skerdi Haviari, Pôle Information Médicale Evaluation Recherche, Hospices Civils de Lyon, Lyon 69003, France; skerdihaviari{at}gmail.com

Abstract

Background Quality improvement and epidemiology studies often rely on database codes to measure performance or impact of adjusted risk factors, but how validity issues can bias those estimates is seldom quantified.

Objectives To evaluate whether and how much interhospital administrative coding variations influence a typical performance measure (adjusted mortality) and potential incentives based on it.

Design National cross-sectional study comparing hospital mortality ranking and simulated pay-for-performance incentives before/after recoding discharge abstracts using medical records.

Setting Twenty-four public and private hospitals located in France

Participants All inpatient stays from the 78 deadliest diagnosis-related groups over 1 year.

Interventions Elixhauser and Charlson comorbidities were derived, and mortality ratios were computed for each hospital. Thirty random stays per hospital were then recoded by two central reviewers and used in a Bayesian hierarchical model to estimate hospital-specific and comorbidity-specific predictive values. Simulations then estimated shifts in adjusted mortality and proportion of incentives that would be unfairly distributed by a typical pay-for-performance programme in this situation.

Main outcome measures Positive and negative predictive values of routine coding of comorbidities in hospital databases, variations in hospitals’ mortality league table and proportion of unfair incentives.

Results A total of 70 402 hospital discharge abstracts were analysed, of which 715 were recoded from full medical records. Hospital comorbidity-level positive predictive values ranged from 64.4% to 96.4% and negative ones from 88.0% to 99.9%. Using Elixhauser comorbidities for adjustment, 70.3% of hospitals changed position in the mortality league table after correction, which added up to a mean 6.5% (SD 3.6) of a total pay-for-performance budget being allocated to the wrong hospitals. Using Charlson, 61.5% of hospitals changed position, with 7.3% (SD 4.0) budget misallocation.

Conclusions Variations in administrative data coding can bias mortality comparisons and budget allocation across hospitals. Such heterogeneity in data validity may be corrected using a centralised coding strategy from a random sample of observations.

  • pay for performance
  • information technology
  • mortality (standardized mortality ratios)
  • performance measures
  • statistics

Statistics from Altmetric.com

Footnotes

  • Contributors AD and CC initiated and oversaw the project. SH and AD designed the analysis plan. AD, FC, SP, CP and AB obtained and pre-processed the data. SH, FC and SP analysed the data. SH and AD wrote the paper. All authors reviewed the paper.

  • Funding This study was funded by the French Ministry of Health.

  • Disclaimer The funding source had no involvement in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. Researchers were independent from the funder.

  • Competing interests None declared.

  • Patient consent Not required.

  • Ethics approval This study was approved by the National Data Protection Commission (Commission Nationale de l’Informatique et des Libertés), in accordance with French ethical directives. We attest that we have obtained appropriate permissions and paid any required fees for use of copyright protected materials.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Original data can be shared with institutions that are authorised by the French Ministry of Health. Simulation results are available upon request.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.