Table 1

Measurement sources for assessment of Choosing Wisely initiatives

Measurement area
Measurement toolsTime frame for Choosing Wisely impactAdvantages of approachDisadvantages of approach
Provider attitudes and awareness
 Physician surveysShort termInexpensivePotentially low response rates
Can reach large number of physiciansMay lack detailed responses
Tailor questions for specific physician groupsMay not be correlated with actual behaviour
Can resample over time to assess change
Can compare physicians response across various health systems
 Structured interviewsShort termMore detailed information about attitudesExpensive and time consuming
May expose factors that can be used to improve campaign
Provider ordering behaviour: overuse of low-value services
 Administrative databasesMedium to long termDirect measurements of overuse at the individual patient levelLack of clinical details in records (eg, treatment indications) may limit specificity of measures and preclude measuring more complex recommendations
Population-level data on utilisation
  As well as measuring the ‘index’ low-value care event, enables measurement of downstream effects (eg, harms and side effects) 
Can provide regional variation on overuse
  Can compare across health systems 
Can reassess over time to assess change
Can monitor for unintended consequences such as underuse
 Electronic health record (EHR)/chart dataMedium to long termMore clinical details (past medical history, previous testing) to assess appropriatenessSome information gaps (ie, symptoms)
Can assess a greater variety of recommendations than with administrative data
As well as measuring the ‘index’ low-value care event, enables measurement of downstream effects (eg, harms and side effects)
Can reassess over time to assess change
Can monitor for unintended consequences so as underuse
Currently can be time consuming and resource intensive; often limited to single centres because of lack of interoperability of EHRs
Patient perceptions and outcomes
 Patient-reported experience measures (PREMs; survey tools)Short to medium termStandardised data collectionNo questions specifically on low-value care
Patient perceptions/drivers could help design interventions to reduce low-value careNot all recommendations directly influenced by patient preference/experience
Impact of interventions on patient perceptions could monitor for unintended consequencesMediating role of cognitive biases (eg, placebo and nocebo effects; cognitive dissonance)
Comparable across jurisdictions
 Patient-reported outcome measures (PROMs; validated survey tools)Medium termValidated patient tools assist providers in developing care plans, including tests and treatmentsNo questions specifically on low-value care
Results can monitor for unintended consequences of interventionsResults may be inappropriately used to ration care