Collecting the data but missing the point: validity of hand hygiene audit data

https://doi.org/10.1016/j.jhin.2015.02.018Get rights and content

Summary

Background

Monitoring of hand hygiene compliance (HHC) by observation has been used in healthcare for more than a decade to provide assurance of infection control practice. The validity of this information is rarely tested.

Aim

To examine the process and validity of collecting and reporting HHC data based on direct observation of compliance.

Methods

Five years of HHC data routinely collected in one large National Health Service hospital trust were examined. The data collection process was reviewed by survey and interview of the auditors. HHC data collected for other research purposes undertaken during this period were compared with the organizational data set.

Findings

After an initial increase, the reported HHC remained unchanged close to its intended target throughout this period. Examination of the data collection process revealed changes, including local interpretations of the data collection system, which invalidated the results. A minority of auditors had received formal training in observation and feedback of results.

Conclusion

Whereas observation of HHC is the current gold standard, unless data collection definitions and methods are unambiguous, published, carefully supervised, and regularly monitored, variations may occur which affect the validity of the data. If the purpose of HHC monitoring is to improve practice and minimize transmission of infection, then a focus on progressively improving performance rather than on achieving a target may offer greater opportunities to achieve this.

Introduction

Optimal hand hygiene practice is recognized as a key infection prevention and transmission intervention.1 Hand hygiene compliance (HHC) in healthcare has traditionally been low.2, 3 Observation, audit, and feedback of performance combined with education and the introduction of alcohol hand rub have been successfully used to increase compliance, although improvement is not always sustained.1, 4, 5, 6, 7, 8 HHC monitoring by direct observation has subsequently become regarded as ‘gold standard’ for measuring and reporting compliance.9, 10 Coupled with feedback, it is established practice in English hospitals where participation in the ‘cleanyourhands’ campaign included the mandatory monitoring and feedback of hand hygiene compliance by direct observation.11, 12 Many healthcare organizations currently collect and report this information to provide assurance of HHC. The expectation is that compliance will be high, and 100% compliance is a widespread aspiration.

This article examines the validity of the HHC data and the data collection method in one healthcare organization for a period of five years. Validity has internal and external components. Internal validity – the subject matter of this article – refers to the ability of the test to accurately measure what is required avoiding bias and error. External validity relates to the generalizability or extrapolation of results.13 There are recognized limitations associated with measuring observed behaviour including the Hawthorne effect, observer bias, and observer drift.14, 15, 16 Such limitations may have a significant effect on the accuracy of the data collected and reported, especially if observers are not trained and monitored.17

This study focuses on an acute hospital with >900 beds, which had increased to >1000 beds by 2012. In 2004, alcoholic hand decontamination was introduced into the hospital, and in 2005 the mean HHC of the organization was <20%. In 2008, the organization-wide routine HHC-monitoring process commenced. In the next three years, >4000 staff were trained in hand hygiene practice and handwashing facilities were improved. At the end of 2008, the mean HHC for the organization had increased to 78% (9328 handwashing events for 11,954 hand-cleansing opportunities) with some diversity in specialties. By this time, >100 staff (predominantly nurses) from 51 wards and departments were trained to undertake direct observation of HHC and to provide feedback of performance to staff. The compliance tool used throughout the organization was adapted from the Lewisham tool and incorporated the World Health Organization's ‘five moments of hand hygiene’.18, 19 Hand hygiene opportunities were categorized as low, medium, and high risk. The organization set an improvement target of 85%, rising by 5% annually until 100% was achieved. The hand hygiene policy was revised to clarify expectations and consequences of non-compliance, which included disciplinary action.

By 2012, the number of wards and departments consistently or intermittently submitting monthly HHC data rose to a maximum of 98, which included all eligible areas, and the number of yearly hand-cleansing opportunities had more than trebled to 42,143. The results were reported, with other organizational performance metrics, in a departmental score card and reviewed monthly by the executive board. Wards and departments were encouraged to display the latest performance data locally on infection control notice boards situated in public areas.

In 2012 the mean HHC for the organization was 94% and reporting compliance was 89% (1031 reports returned, of 1164 requested). Areas with scores below a lower threshold (85% in 2012) were followed up by the infection control team and managers were alerted. However, the reported data were at times at odds with observed practice, feedback from service users, and with random checks. This prompted an examination of the methodology and validity of the data collection process.

Section snippets

Methods

Several methods were used to obtain a more comprehensive picture of the data collection process. These included:

  • examination of five years of organization-wide HHC data, including comparison with HHC research data collected during the same period;

  • survey and interview of staff collecting routine HHC data;

  • comparison of product usage (this method was abandoned).

Routine data: number of hand hygiene opportunities

The reported median number of hand hygiene opportunities peaked in 2009 and remained high over the remaining period (Table I), although this number varied widely between reporting locations, some of which reported zero opportunities (making it impossible to assess hand hygiene compliance), others reported as many as 615 opportunities. Some locations showed significant declines in the average number of observed hand hygiene opportunities (Table II). The neonatal unit observed the opposite trend,

Discussion

In this study we investigated the validity of hand hygiene monitoring by direct observation. Despite widespread use of this method, ours is one of the first studies to examine how well it is undertaken, although only in one organization. We found that HHC rates reported through routine HHC data were much higher than those obtained through in-depth research studies. Interviews with staff indicated that the validity of routine HHC data may be compromised by lack of staff training, inconsistency

Acknowledgements

We thank C. Narsico for undertaking the interviews.

References (26)

  • B.C. Lam et al.

    Hand hygiene practices in a neonatal intensive care unit: a multimodal intervention and impact on nosocomial infection

    Pediatrics

    (2004)
  • H. Aboumatar et al.

    Infection prevention promotion program based on the PRECEDE model: improving hand hygiene behaviors among healthcare personnel

    Infect Control Hosp Epidemiol

    (2012)
  • D.J. Gould et al.

    Interventions to improve hand hygiene compliance in patient care

    Cochrane Database Syst Rev

    (2011)
  • Cited by (0)

    View full text