You are here

  • Home
  • Archive
  • Volume 24, Issue 12
  • Development of the Quality Improvement Minimum Quality Criteria Set (QI-MQCS): a tool for critical appraisal of quality improvement intervention publications

Article Text

Article menu
Research and reporting methodology
Development of the Quality Improvement Minimum Quality Criteria Set (QI-MQCS): a tool for critical appraisal of quality improvement intervention publications
  1. Susanne Hempel1,
  2. Paul G Shekelle1,2,
  3. Jodi L Liu1,
  4. Margie Sherwood Danz1,3,
  5. Robbie Foy4,
  6. Yee-Wei Lim5,
  7. Aneesa Motala1,
  8. Lisa V Rubenstein1,3,6
  1. 1RAND Corporation, Santa Monica, California, USA
  2. 2Veterans Affairs West Los Angeles Medical Center, Los Angeles, California, USA
  3. 3Veterans Affairs Greater Los Angeles, North Hills, California, USA
  4. 4University of Leeds, Leeds Institute of Health Sciences, Leeds, UK
  5. 5National University of Singapore, Saw Swee Hock School of Public Health, Singapore
  6. 6University of California, Department of Medicine, Los Angeles, California, USA
  1. Correspondence to Dr Susanne Hempel, RAND Health, RAND Corporation, 1776 Main Street, Santa Monica, CA 90407, USA; susanne_hempel{at}rand.org

Abstract

Objective Valid, reliable critical appraisal tools advance quality improvement (QI) intervention impacts by helping stakeholders identify higher quality studies. QI approaches are diverse and differ from clinical interventions. Widely used critical appraisal instruments do not take unique QI features into account and existing QI tools (eg, Standards for QI Reporting Excellence) are intended for publication guidance rather than critical appraisal. This study developed and psychometrically tested a critical appraisal instrument, the QI Minimum Quality Criteria Set (QI-MQCS) for assessing QI-specific features of QI publications.

Methods Approaches to developing the tool and ensuring validity included a literature review, in-person and online survey expert panel input, and application to empirical examples. We investigated psychometric properties in a set of diverse QI publications (N=54) by analysing reliability measures and item endorsement rates and explored sources of disagreement between reviewers.

Results The QI-MQCS includes 16 content domains to evaluate QI intervention publications: Organisational Motivation, Intervention Rationale, Intervention Description, Organisational Characteristics, Implementation, Study Design, Comparator Description, Data Sources, Timing, Adherence/Fidelity, Health Outcomes, Organisational Readiness, Penetration/Reach, Sustainability, Spread and Limitations. Median inter-rater agreement for QI-MQCS items was κ 0.57 (83% agreement). Item statistics indicated sufficient ability to differentiate between publications (median quality criteria met 67%). Internal consistency measures indicated coherence without excessive conceptual overlap (absolute mean interitem correlation=0.19). The critical appraisal instrument is accompanied by a user manual detailing What to consider, Where to look and How to rate.

Conclusions We developed a ready-to-use, valid and reliable critical appraisal instrument applicable to healthcare QI intervention publications, but recognise scope for continuing refinement.

  • Quality improvement
  • Evaluation methodology
  • Evidence-based medicine
  • Healthcare quality improvement
  • Quality improvement methodologies

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjqs-2014-003151

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.