Overview of methodological, contextual and managerial fitness for use considerations
Clusters | Considerations | Guiding questions for considering an indicator’s use |
Methodological | ||
Measures what matters. | Does anybody care? | |
Wide engagement. | What can we do? | |
Easily interpreted. | Does the indicator signal a clear direction? | |
Clear standardisation. | Is the indicator clearly defined and replicable? | |
Alignment of accountability. | Are entry points for taking action feasible? | |
Measurement matches delivery. | Is the indicator a reflection of the system? | |
Sensitive to meaningful change. | Is the indicator sufficiently sensitive to change? | |
Contextual | ||
Information infrastructure | Interoperability. | Can needed data be accessed? |
Data quality. | Is the data of quality? | |
Governance | Political will and vision. | Is there high-level commitment and direction for use? |
Regulation for data protection. | Does existing legislation facilitate use? | |
Cross-sector partnerships. | Are cross-sector partnerships in place? | |
Aligned financing structures. | Do financing structures encourage the intended use? | |
Workforce capacity | Data and quality expertise. | Are the competencies to interpret and use data in place? |
Time dedicated to improvement. | Is time allocated to encourage use? | |
Culture | Learning orientation. | Is an environment for learning cultivated? |
Shared responsibility for health. | Do users feel accountable for improvement? | |
Managerial | ||
Selecting healthcare performance indicators | Clear purpose of use. | What is the purpose of use? (eg, strategy development) |
Target end user is known. | Is the target audience known? (eg, clinicians, public) | |
Conceptual framework. | Is the dimension of quality pursued clear? | |
Indicator quality. | Is the indicator scientifically sound? | |
Source, type and availability of data. | What data are needed and are they available? (eg, administrative, clinical, survey data, wearables) | |
Standards for appraisal. | How will improvements in performance be assessed? | |
Degree of public disclosure. | Is the indicator for internal or external (public) use? | |
Accompanying indicators. | Are there relevant accompanied indicators? | |
Previous use. | Has the indicator been used previously? | |
Accessing data | Representativeness of data. | Are the data complete? |
Data linkages. | Can relevant data sources be linked? | |
Data collection tools. | How will data be collected? (eg, paper-based, automated electronically, manual electronic entry) | |
Unity of language/coding. | Is there consistency in coding across data to be used? | |
Applying methods of analysis | Type of analysis. | How will the data be analysed? (eg, benchmarking, time trend, case mix correction) |
Aggregation of indicators. | How can composites/indices be used to simplify data? | |
Reference group. | Who is the reference group? | |
Breakdowns/cohorts. | How will the data be disaggregated? (eg, age, sex, ethnicity, geographically) | |
Calculation of values. | How will values be calculated? (eg, mean, median, SD, top 10% mean) | |
Time interval. | Should a time trend be reported and at what interval? | |
Application of risk adjustments. | How will risk adjustments be applied? (eg, variable specification, source, weighting scheme) | |
Managing missing data. | How will missed data points be handled? | |
Contextualising data. | What other data are needed to give the indicator meaning? | |
Displaying findings | Chart options. | How will the data be visualised? (eg, chart, map, table) |
Simplification techniques. | What techniques to simplify the meaning can be applied? (eg, colour, size variation, icons) | |
Customisation of display. | How can users customise the data? (eg, change of display, change of information) | |
Narrated interpretation. | How can the quality and the meaning of data be narrated? | |
Format of reporting. | How will it be reported? (eg, print, mobile, web-based) | |
Reaching decision-makers | Frequency of reporting. | What is the relevant reporting cycle (eg, real time, quarterly, annually, biennially) |
Dissemination channels. | How will users be reached? (eg, mail, email, champions) | |
Guidance on use. | How can users be supported to make use of findings? |