Article Text

Download PDFPDF

Impact of audit and feedback with action implementation toolbox on improving ICU pain management: cluster-randomised controlled trial
  1. Marie-José Roos-Blom1,2,
  2. Wouter T Gude1,
  3. Evert de Jonge2,3,
  4. Jan Jaap Spijkstra2,4,
  5. Sabine N van der Veer5,
  6. Niels Peek5,6,
  7. Dave A Dongelmans2,7,
  8. Nicolette F de Keizer1,2
  1. 1 Department of Medical Informatics, Amsterdam Public Health Research Institute, Amsterdam UMC, University of Amsterdam, Amsterdam, The Netherlands
  2. 2 National Intensive Care Evaluation (NICE) Foundation, Amsterdam, The Netherlands
  3. 3 Department of Intensive Care Medicine, Leiden University Medical Center, Leiden, The Netherlands
  4. 4 Department of Intensive Care Medicine, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
  5. 5 Centre for Health Informatics, Division of Informatics, Imaging and Data Sciences, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
  6. 6 NIHR Greater Manchester Patient Safety Translational Research Centre, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
  7. 7 Department of Intensive Care Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, The Netherlands
  1. Correspondence to Marie-José Roos-Blom, Department of Medical Informatics, Amsterdam Public Health research institute, Amsterdam UMC, University of Amsterdam, Amsterdam 1081 HV, The Netherlands; m.blom{at}


Background Audit and feedback (A&F) enjoys widespread use, but often achieves only marginal improvements in care. Providing recipients of A&F with suggested actions to overcome barriers (action implementation toolbox) may increase effectiveness.

Objective To assess the impact of adding an action implementation toolbox to an electronic A&F intervention targeting quality of pain management in intensive care units (ICUs).

Trial design Two-armed cluster-randomised controlled trial. Randomisation was computer generated, with allocation concealment by a researcher, unaffiliated with the study. Investigators were not blinded to the group assignment of an ICU.

Participants Twenty-one Dutch ICUs and patients eligible for pain measurement.

Interventions Feedback-only versus feedback with action implementation toolbox.

Outcome Proportion of patient-shift observations where pain management was adequate; composed by two process (measuring pain at least once per patient in each shift; re-measuring unacceptable pain scores within 1 hour) and two outcome indicators (acceptable pain scores; unacceptable pain scores normalised within 1 hour).

Results 21 ICUs (feedback-only n=11; feedback-with-toolbox n=10) with a total of 253 530 patient-shift observations were analysed. We found absolute improvement on adequate pain management in the feedback-with-toolbox group (14.8%; 95% CI 14.0% to 15.5%) and the feedback-only group (4.8%; 95% CI 4.2% to 5.5%). Improvement was limited to the two process indicators. The feedback-with-toolbox group achieved larger effects than the feedback-only group both on the composite adequate pain management (p<0.05) and on measuring pain each shift (p<0.001). No important adverse effects have occurred.

Conclusion Feedback with toolbox improved the number of shifts where patients received adequate pain management compared with feedback alone, but only in process and not outcome indicators.

Trial registration number NCT02922101.

  • pain
  • feedback
  • intensive care units
  • quality improvement
  • dashboard
  • action implementation toolbox

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See:

View Full Text

Statistics from


  • M-JR-B and WTG are joint first authors.

  • Contributors All authors contributed to study conception and participated in critically appraising and revising the intellectual content of the manuscript. M-JR-B and WTG were primarily responsible for the data collection and analyses and the manuscript draft. All authors read and approved the final manuscript.

  • Funding The NICE Foundation arranged the development of the quality indicators and funded the development of the quality dashboard.

  • Competing interests The National Intensive Care Evaluation (NICE) Foundation pays the Department of Medical Informatics, Academic Medical Center for processing, checking and maintaining the Dutch quality registry. M-JR-B, WTG and NFdK are employees of the Department of Medical Informatics and work for the NICE registry. NFdK, DAD, EdJ and JJS are members of the NICE board.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.