Article Text
PDF
Abstract
Background Healthcare quality measurement systems, which use aggregated patient-level quality measures to assess organisational performance, have been introduced widely. Yet, their usefulness in practice has received scant attention. Using Minnesota nursing home quality indicators (QIs) as a case example, we demonstrate an approach for systematically evaluating QIs in practice based on: (a) parsimony and relevance, (b) usability in discriminating between facilities, (c) actionability and (d) construct validity.
Methods We analysed 19 risk-adjusted, facility-level QIs over the 2012–2019 period. Parsimony and relevance of QIs were evaluated using scatter plots, Pearson correlations, literature review and expert opinions. Discrimination between facilities was assessed by examining facility QI distributions and the impact of the distributions on scoring. Actionability of QIs was assessed through QI trends over time. Construct validity was assessed through exploratory factor analysis of domain structure for grouping the QIs.
Results Correlation analysis and qualitative assessment led to redefining one QI, adding one improvement-focused QI, and combining two highly correlated QIs to improve parsimony and clinical relevance. Ten of the QIs displayed normal distributions which discriminated well between the best and worst performers. The other nine QIs displayed poor discrimination; they had skewed distributions with ceiling or floor effects. We recommended scoring approaches tailored to these distributions. One QI displaying substantial improvement over time was recommended for retirement (physical restraint use). Based on factor analysis, we grouped the 18 final QIs into four domains: incontinence (4 QIs), physical functioning (4 QIs), psychosocial care (4 QIs) and care for specific conditions (6 QIs).
Conclusion We demonstrated a systematic approach for evaluating QIs in practice by arriving at parsimonious and relevant QIs, tailored scoring to different QI distributions and a meaningful domain structure. This approach could be applied in evaluating quality measures in other health or long-term care settings.
- Nursing homes
- Quality measurement
- Evaluation methodology
Data availability statement
Data may be obtained from a third party and are not publicly available.
Statistics from Altmetric.com
Data availability statement
Data may be obtained from a third party and are not publicly available.
Footnotes
Contributors All authors have read and approved the submission of this manuscript. DX and GA contributed to the study concept and design, acquisition, analysis and interpretation of data, and preparation of the manuscript. TL and MR contributed to data interpretation and manuscript preparation. DX was responsible for statistical analysis and the overall content as the guarantor.
Funding This study was funded by evaluation contract with the Minnesota Department of Human Services.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.