Round 1: Pre-selection | Selecting key recommendations | Small group of quality indicators developers (1–3 persons) | Outcome of care:
• Patients’ health (morbidity, mortality, health status)
• Cost |
Round 2: Rating and adding | Rating and adding key recommendations | Expert panel (8–10 persons) | • Patients’ health
• Cost
• Sensitivity to change
• Availability of data |
Round 3: Reliability | Determining inter- and intra-rater reliability | Expert panel for the rating Research team for the analyses | • Kappa, rho |
Round 4: Potential indicators | Getting set of potential indicators | Research team | • Cut off score: mean above mid of rating scale
• Agreement among 80% of the panel members |
Round 5: Reflection | Acceptability of indicators | Research team Laymen professionals | • Face validity |