Collecting the data but missing the point: validity of hand hygiene audit data
Introduction
Optimal hand hygiene practice is recognized as a key infection prevention and transmission intervention.1 Hand hygiene compliance (HHC) in healthcare has traditionally been low.2, 3 Observation, audit, and feedback of performance combined with education and the introduction of alcohol hand rub have been successfully used to increase compliance, although improvement is not always sustained.1, 4, 5, 6, 7, 8 HHC monitoring by direct observation has subsequently become regarded as ‘gold standard’ for measuring and reporting compliance.9, 10 Coupled with feedback, it is established practice in English hospitals where participation in the ‘cleanyourhands’ campaign included the mandatory monitoring and feedback of hand hygiene compliance by direct observation.11, 12 Many healthcare organizations currently collect and report this information to provide assurance of HHC. The expectation is that compliance will be high, and 100% compliance is a widespread aspiration.
This article examines the validity of the HHC data and the data collection method in one healthcare organization for a period of five years. Validity has internal and external components. Internal validity – the subject matter of this article – refers to the ability of the test to accurately measure what is required avoiding bias and error. External validity relates to the generalizability or extrapolation of results.13 There are recognized limitations associated with measuring observed behaviour including the Hawthorne effect, observer bias, and observer drift.14, 15, 16 Such limitations may have a significant effect on the accuracy of the data collected and reported, especially if observers are not trained and monitored.17
This study focuses on an acute hospital with >900 beds, which had increased to >1000 beds by 2012. In 2004, alcoholic hand decontamination was introduced into the hospital, and in 2005 the mean HHC of the organization was <20%. In 2008, the organization-wide routine HHC-monitoring process commenced. In the next three years, >4000 staff were trained in hand hygiene practice and handwashing facilities were improved. At the end of 2008, the mean HHC for the organization had increased to 78% (9328 handwashing events for 11,954 hand-cleansing opportunities) with some diversity in specialties. By this time, >100 staff (predominantly nurses) from 51 wards and departments were trained to undertake direct observation of HHC and to provide feedback of performance to staff. The compliance tool used throughout the organization was adapted from the Lewisham tool and incorporated the World Health Organization's ‘five moments of hand hygiene’.18, 19 Hand hygiene opportunities were categorized as low, medium, and high risk. The organization set an improvement target of 85%, rising by 5% annually until 100% was achieved. The hand hygiene policy was revised to clarify expectations and consequences of non-compliance, which included disciplinary action.
By 2012, the number of wards and departments consistently or intermittently submitting monthly HHC data rose to a maximum of 98, which included all eligible areas, and the number of yearly hand-cleansing opportunities had more than trebled to 42,143. The results were reported, with other organizational performance metrics, in a departmental score card and reviewed monthly by the executive board. Wards and departments were encouraged to display the latest performance data locally on infection control notice boards situated in public areas.
In 2012 the mean HHC for the organization was 94% and reporting compliance was 89% (1031 reports returned, of 1164 requested). Areas with scores below a lower threshold (85% in 2012) were followed up by the infection control team and managers were alerted. However, the reported data were at times at odds with observed practice, feedback from service users, and with random checks. This prompted an examination of the methodology and validity of the data collection process.
Section snippets
Methods
Several methods were used to obtain a more comprehensive picture of the data collection process. These included:
- –
examination of five years of organization-wide HHC data, including comparison with HHC research data collected during the same period;
- –
survey and interview of staff collecting routine HHC data;
- –
comparison of product usage (this method was abandoned).
Routine data: number of hand hygiene opportunities
The reported median number of hand hygiene opportunities peaked in 2009 and remained high over the remaining period (Table I), although this number varied widely between reporting locations, some of which reported zero opportunities (making it impossible to assess hand hygiene compliance), others reported as many as 615 opportunities. Some locations showed significant declines in the average number of observed hand hygiene opportunities (Table II). The neonatal unit observed the opposite trend,
Discussion
In this study we investigated the validity of hand hygiene monitoring by direct observation. Despite widespread use of this method, ours is one of the first studies to examine how well it is undertaken, although only in one organization. We found that HHC rates reported through routine HHC data were much higher than those obtained through in-depth research studies. Interviews with staff indicated that the validity of routine HHC data may be compromised by lack of staff training, inconsistency
Acknowledgements
We thank C. Narsico for undertaking the interviews.
References (26)
- et al.
Effectiveness of a hospital-wide programme to improve compliance with hand hygiene. Infection Control Programme
Lancet
(2000) - et al.
Monitoring and feedback of hand hygiene compliance and the impact on facility-acquired methicillin-resistant Staphylococcus aureus
Am J Infect Control
(2008) - et al.
Bias and causal associations in observational research
Lancet
(2002) - et al.
Routine hand hygiene audit by direct observation: has nemesis arrived?
J Hosp Infect
(2011) - et al.
‘My five moments for hand hygiene’: a user-centred design approach to understand, train, monitor and report hand hygiene
J Hosp Infect
(2007) - et al.
Hand hygiene after touching a patient's surroundings: the opportunities most commonly missed
J Hosp Infect
(2013) - et al.
Understanding adherence to hand hygiene recommendations: the theory of planned behaviour
Am J Infect Control
(2001) - et al.
Hand-washing patterns in medical intensive-care units
N Engl J Med
(1981) Teaching hospital medical staff to handwash
Med J Aust
(1996)- et al.
Alcohol-based handrub improves compliance with hand hygiene in intensive care units
Archs Intern Med
(2002)