Article Text
PDF
Abstract
Background As quality improvement (QI) programmes have become progressively larger scale, the risks of implementation having unintended consequences are increasingly recognised. More routine use of balancing measures to monitor unintended consequences has been proposed to evaluate overall effectiveness, but in practice published improvement interventions hardly ever report identification or measurement of consequences other than intended goals of improvement.
Methods We conducted 15 semistructured interviews and two focus groups with 24 improvement experts to explore the current understanding of balancing measures in QI and inform a more balanced accounting of the overall impact of improvement interventions. Data were analysed iteratively using the framework approach.
Results Participants described the consequences of improvement in terms of desirability/undesirability and the extent to which they were expected/unexpected when planning improvement. Four types of consequences were defined: expected desirable consequences (goals); expected undesirable consequences (trade-offs); unexpected undesirable consequences (unpleasant surprises); and unexpected desirable consequences (pleasant surprises). Unexpected consequences were considered important but rarely measured in existing programmes, and an improvement pause to take stock after implementation would allow these to be more actively identified and managed. A balanced accounting of all consequences of improvement interventions can facilitate staff engagement and reduce resistance to change, but has to be offset against the cost of additional data collection.
Conclusion Improvement measurement is usually focused on measuring intended goals, with minimal use of balancing measures which when used, typically monitor trade-offs expected before implementation. This paper proposes that improvers and leaders should seek a balanced accounting of all consequences of improvement across the life of an improvement programme, including deliberately pausing after implementation to identify and quantitatively or qualitatively evaluate any pleasant or unpleasant surprises.
- healthcare quality improvement
- quality measurement
- qualitative research
- patient safety
- implementation science
Statistics from Altmetric.com
Footnotes
Contributors MT and BG were responsible for planning the study and led the data collection and analysis. TD, NMG and DC contributed to data analysis. MT drafted and led the writing of the manuscript. BG, TD, NMG and DC participated in critically appraising and revising the intellectual content of the manuscript. All authors read and approved the final manuscript.
Funding This paper presents core work from the Scottish Improvement Science Collaborating Centre, funded by the Scottish Funding Council, Health Foundation, Chief Scientist Office, and NHS Education Scotland with in-kind contributions from participating partner universities and health boards. The funding bodies had no role in the design of the study and collection, analysis and interpretation of data, and in writing the manuscript.
Competing interests None declared.
Patient consent Detail has been removed from this case description/these case descriptions to ensure anonymity. The editors and reviewers have seen the detailed information available and are satisfied that the information backs up the case the authors are making.
Ethics approval The University of Dundee Research Ethics Committee granted ethical approval for this study (UREC 15069).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The data used and/or analysed during the current study are available from the corresponding author on reasonable request.