Article Text

PDF
A systematic review of behavioural marker systems in healthcare: what do we know about their attributes, validity and application?
  1. Aaron S Dietz1,
  2. Peter J Pronovost1,2,
  3. Kari N Benson1,
  4. Pedro Alejandro Mendez-Tellez2,
  5. Cynthia Dwyer3,
  6. Rhonda Wyskiel1,
  7. Michael A Rosen1,2
  1. 1The Armstrong Institute for Patient Safety and Quality, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
  2. 2Department of Anesthesiology and Critical Care Medicine, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
  3. 3Surginal Intensive Care Unit, Johns Hopkins Hospital, Baltimore, Maryland, USA
  1. Correspondence to Dr Michael A Rosen, Armstrong Institute for Patient Safety and Quality, and Department of Anesthesiology & Critical Care Medicine, Johns Hopkins University School of Medicine, 750 East Pratt Street, 15th Floor, Baltimore, MD 21202, USA; mrosen44{at}jhmi.edu

Abstract

Objective Behavioural marker systems are advocated as a method for providing accurate assessments, directing feedback and determining the impact of teamwork improvement initiatives. The present article reports on the state of quality surrounding their use in healthcare and discusses the implications of these findings for future research, development and application. In doing so, this article provides a practical resource where marker systems can be selected and evaluated based on their strengths and limitations.

Methods Four research questions framed this review: what are the attributes of behavioural marker systems? What evidence of reliability and validity exists? What skills and expertise are required for their use? How have they been applied to investigate the relationship between teamwork and other constructs?

Results Behavioural markers systems are generally designed for specific work domains or tasks. They often cover similar content with inconsistent terminology, which complicates the comparison of research findings across clinical domains. Although several approaches were used to establish the reliability and validity of marker systems, the marker system literature, as a whole, requires more robust reliability and validity evidence. The impact of rater training on rater proficiency was mixed, but evidence suggests that improvements can be made over time.

Conclusions A consensus of definitions for teamwork constructs must be reached to ensure that the meaning behind behavioural measurement is understood across disciplines, work domains and task types. Future development efforts should focus on the cost effectiveness and feasibility of measurement tools including time spent training raters. Further, standards for the testing and reporting of psychometric evidence must be established. Last, a library of tools should be generated around whether the instrument measures general or domain-specific behaviours.

  • Teamwork
  • Qualitative research
  • Performance measures

Statistics from Altmetric.com

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.