Article Text

Applying thematic synthesis to interpretation and commentary in epidemiological studies: identifying what contributes to successful interventions to promote hand hygiene in patient care
  1. Nicholas Drey1,
  2. Dinah Gould1,
  3. Edward Purssell1,
  4. Jane Chudleigh1,
  5. Donna Moralejo2,
  6. Rose Gallagher3,
  7. Annette Jeanes4,
  8. Neil Wigglesworth5,
  9. Didier Pittet6
  1. 1 School of Health Sciences, City, University of London, London, United Kingdom
  2. 2 Memorial University of Newfoundland, St. John's, Newfoundland and Labrador, Canada
  3. 3 Royal College of Nursing, London, London, UK
  4. 4 Independent Consultant, London, United Kingdom
  5. 5 Guy's and St Thomas' NHS Foundation Trust, London, United Kingdom
  6. 6 Infection Prevention and Control Programme, University of Geneva Hospitals and Faculty of Medicine, Geneva, Switzerland
  1. Correspondence to Dr Edward Purssell, School of Health Sciences, City University, London EC1V 0HB, UK; edward.purssell{at}city.ac.uk

Abstract

Objectives Hand hygiene is considered the most important preventive measure for healthcare-associated infections, but adherence is suboptimal. We previously undertook a Cochrane Review that demonstrated that interventions to improve adherence are moderately effective. Impact varied between organisations and sites with the same intervention and implementation approaches. This study seeks to explore these differences.

Methods A thematic synthesis was applied to the original authors’ interpretation and commentary that offered explanations of how hand hygiene interventions exerted their effects and suggested reasons why success varied. The synthesis used a published Cochrane Review followed by three-stage synthesis.

Results Twenty-one papers were reviewed: 11 randomised, 1 non-randomised and 9 interrupted time series studies. Thirteen descriptive themes were identified. They reflected a range of factors perceived to influence effectiveness. Descriptive themes were synthesised into three analytical themes: methodological explanations for failure or success (eg, Hawthorne effect) and two related themes that address issues with implementing hand hygiene interventions: successful implementation needs leadership and cooperation throughout the organisation (eg, visible managerial support) and understanding the context and aligning the intervention with it drives implementation (eg, embedding the intervention into wider patient safety initiatives).

Conclusions The analytical themes help to explain the original authors’ perceptions of the degree to which interventions were effective and suggested new directions for research: exploring ways to avoid the Hawthorne effect; exploring the impact of components of multimodal interventions; the use of theoretical frameworks for behaviour change; potential to embed interventions into wider patient safety initiatives; adaptations to demonstrate sustainability; and the development of systematic approaches to implementation. Our findings corroborate studies exploring the success or failure of other clinical interventions: context and leadership are important.

  • infection control
  • health services research
  • human factors
  • implementation science

Statistics from Altmetric.com

Background

Healthcare-associated infection (HCAI) is spread mainly via health workers’ hands. Adherence to hand hygiene protocols is suboptimal, and the impact of campaigns to increase adherence to hand hygiene protocols is hard to sustain.1 The WHO’s1 multimodal hand hygiene promotion strategy recommends system change, that is, the use of alcohol-based handrub at the point of care, written and/or verbal reminders, education and audit with performance feedback and the promotion of institutional safety climate in relation to hand hygiene. Other components of the hand hygiene intervention (HHI) can be added or modified to customise core recommendations to local need.1 Initiatives to promote hand hygiene are widely reported but most are uncontrolled before-and-after studies insufficiently robust to generate findings that can be considered sufficiently rigorous to support policy or practice; our recent Cochrane systematic review (second update published in 2017) of the most rigorous interventions2 demonstrated only modest improvement with variations between organisations and different sites in the same organisation when the same intervention and approach to implementation were applied. Our Cochrane review did not investigate factors that might have contributed to differences in effectiveness. We therefore analysed the original authors’ interpretation of and commentary on their findings to explore reasons to explain this variation and identify messages for future research, policy and practice. Two research questions were addressed:

  1. What factors identified by thematic synthesis are perceived by the original authors to influence the effectiveness (or lack of effectiveness) of HHIs in different contexts?

  2. What are the messages for research, policy and practice?

Methods

We took an inductive approach to analysis to generate new insights and understandings of the original authors’ interpretations for the success or failure of HHIs using an adapted approach to thematic synthesis, a method originally developed to bring together and integrate the findings of qualitative studies in healthcare research.3 4

The adapted thematic synthesis was conducted on the systematic searches and quality appraisal previously conducted for a Cochrane Review of HHIs. The data for analysis and synthesis were the individual study authors’ interpretation and commentary offering explanations of how HHIs exerted their effects and the suggested reasons why success varied. The three-stage approach to synthesis remained unchanged from that described by Thomas and Harden.4 It involved line-by-line coding of the information contained in primary studies, its organisation and the development of descriptive themes that remained ‘close to the [primary] data’. The aim of this rigorous process was to create analytical themes in which the reviewers ‘go beyond’ the primary studies to provide explanation and identify messages for practice, policy and future research.3 4

DJG and JC undertook line-by-line coding of the original authors’ accounts of their studies and their opinions of what contributed to or detracted from the effectiveness of the HHI to generate provisional descriptive themes. These were agreed between other members of the research team (ND and EP). The resulting descriptive themes were labelled and synthesised into analytical themes. Membership of the full research team included policymakers, clinicians and academics with experience in qualitative and quantitative analysis in order to increase utility of the findings as recommended.3 DJG, JC, DP, DM, RG, AJ and NW have backgrounds in infection prevention. DJG, DP, RG, AJ and NW have contributed to policy, including policy relating to hand hygiene. EP’s background is in surveillance of infectious diseases. ND is an epidemiologist.

Included publications

Eligibility of papers for the thematic synthesis was based on eligibility to be included in our recently updated Cochrane Review (Cochrane Database of Systematic Reviews 2017, Issue 9. Cochrane Reviews are regularly updated as new evidence emerges and in response to comments and criticisms. The Cochrane Database of Systematic Reviews should be consulted for the most recent version of the Review) that included 26 papers meeting the stringent quality criteria of the Cochrane Evaluation of Practice and Care Group.5 This is an adaptation of the standard approach to thematic synthesis described by Thomas and Harden.4 The approach they recommend would usually be conducted with a smaller purposive sample of qualitative findings. We chose to use a large sample composed of all the eligible papers from the Cochrane Review, because our analysis is conducted on the original authors’ interpretation and commentary of their quantitative findings and not on standard qualitative findings as generated from a typical qualitative study. It would not have been logical to interpret commentary in HHI studies already deemed to be of poor quality, as the validity of the studies’ findings is unknown and thus explanations for their success or otherwise non-sensical. Existing tools employed to critically appraise qualitative work were not applicable to the types of study we were investigating.

To meet the criteria for thematic synthesis, papers had to contain authors’ interpretations and commentary offering explanation of how HHIs exerted their effects and suggest reasons why success varied. Before embarking on thematic synthesis, the text of each publication was scrutinised to determine whether this information was provided. Two members of the research team worked together to select the included publications (DJG and ND). Third party arbitration to resolve divergent opinion was not required.

Five publications were excluded from those originally included in the Cochrane Review.2 All were short reports in which the original authors did not express any opinions about why or how the HHIs were effective. All the excluded publications concluded that the HHI had increased hand hygiene adherence.

Twenty-one publications were included: 11 randomised trials6–16; 1 non-randomised trial17; and 9 interrupted time series (ITS) studies meeting specific criteria adopted by the Cochrane Collaboration.18–26

Summary details of the 21 included studies including the study characteristics and contexts are available in online supplementary table 1 showing study design, journal type, aims, HHI intervention design, method of hand hygiene audit, basis for determining adherence, type of HHI, the study setting, its duration, stakeholder involvement and challenges to recruitment. Online supplementary table 4 contains further information on the characteristics of the included studies. These differ from the Cochrane Review2 as five studies are excluded from thematic synthesis.

Supplemental material

Supplemental material

The HHI was considered effective by 11 research teams according to their own interpretation,7 9–11 14 17 20 22–25 moderately effective by a further five8 13 16 19 26 and disappointing by three research teams6 12 15 21 taking into consideration baseline adherence that in one organisation was already good (66%).15 In one further study, effectiveness was not reported as the effort required to implement the HHI was not considered worthwhile because increase in adherence was modest and did not alter rates of colonisation by meticillin-resistant Staphylococcus aureus (MRSA).12

Results

Provisional descriptive themes

We identified and labelled 13 provisional descriptive themes (table 1 and detail in online supplementary table 2). We established that there was broad agreement between the different studies in terms of original authors’ opinions. Similar descriptive themes were apparent in many of the studies. For example, the descriptive theme: ‘Concerns about the Hawthorne effect and controlling for bias’ emerged in all but 2 publications6 8 9 11 12 14 16–18 20 24 and the descriptive theme: ‘Seeking and obtaining organisational support for HHI is important but not always successful’ also appeared in a number of publications. Attempts to obtain organisational support to promote HHIs were made in 11 publications but with variable success.6–9 12 13 22–26 Some divergences of the original authors’ opinion were apparent, for example, in the descriptive theme: ‘HHIs work differently in different clinical settings and with different groups’. HHIs were reported to work better in some clinical settings than others.6 16 26 Not all the original authors believed that this heterogeneity was problematic however.19

Supplemental material

Table 1

The analytical and descriptive themes with examples of supporting evidence extracted from the primary studies

Analytical themes

The descriptive themes were synthesised into three analytical themes through discussion and reflection. The themes developed in this way were designed to capture the meaning and content of the findings accurately, discretely and succinctly without recourse to unnecessary extraneous themes. Three major analytic themes emerged: methodological explanations for failure or success of the study; and two related themes that address issues with implementing HHIs: successful implementation needs leadership and cooperation from throughout the organisation and understanding the context and aligning the intervention with it drives implementation. Table 1 presents the analytical themes and how the descriptive themes map onto them with exemplars supporting evidence extracted from the primary studies. Further detail is available in online supplementary table 3.

Supplemental material

Methodological explanations for failure or success

In 20 of the 21 (95%) publications,6–25 methodological limitations were perceived by the original authors to impinge on their ability to demonstrate the effectiveness of the HHI in their particular study. Concerns about the Hawthorne effect (increased hand hygiene adherence when health workers become aware that they are watched)27 were prominent. Other biases affecting the internal validity of studies, which were difficult to eliminate in the study design were also identified. These included cross-contamination with control areas. Four studies6 12 13 15 identified the problem of showing a meaningful increase in hand hygiene adherence in organisations where adherence was already high. Bundled interventions also posed a problem for researchers wanting to establish which particular elements of a multimodal intervention were responsible for improvements in adherence.23 24 Methodological factors were not always cited as limitations by the original authors. A number of unique features were credited with ability to demonstrate sustained benefit. These included using a stepped wedge design, extended duration of the HHI6 and feedback of audit findings.13 15 18 A number of these authors also identified having a theoretical underpinning to the HHI as factor contributing to its success.6 7 9 10

Successful implementation needs leadership and cooperation from throughout the organisation

Leadership was widely cited as essential to the implementation of HHIs.6 7 13 22 24 25 In particular, high visibility of managerial and senior clinical staff was important to the necessary change of cultural practices and behaviour22 25 and developing a consistent and sustained approach to hand hygiene adherence.13 22 24 Approaches where particular roles were embedded within training and faculty positions also promoted successful implementation.25 Absence of ‘buy-in’ from health workers was cited as a reason for poor implementation.6 The potential role of patients in securing hand hygiene adherence was acknowledged by two authors, but the experience proved problematic as patients were unwilling to challenge healthcare workers.8 15

A number of the original authors recognised the importance of having a flexible approach to the HHI to enable it able to fit in with the needs of specific groups of staff and specific clinical settings, often in multiple hospital sites.21 23 26 High staff turnover was recognised as problematic and frequent feedback sessions were employed to ensure that new employees were ‘brought up to speed’ quickly.16 In other studies, the HHI was designed with local input from clinical staff to ensure ‘buy-in’ and motivation.26

Understanding the context and aligning the intervention with it drives implementation

‘Buy-in’ from staff and organisational support was not the only factor determining the success of HHIs. Equally important was the need to understand the specific context of the HHI and to align it to this context to ensure successful implementation. Strategies that embedded the HHI in existing patient safety and quality improvement initiatives were seen as successful,14 23 25 particularly where expertise could be shared with larger units.14 Allied to this was the need for the HHI to be acceptable to health workers and for them to be included in behaviour change modifications. Interventions that enable managers to address other patient safety issues were identified as helpful.9 Not involving health workers, disinterest or resistance to the HHI were barriers to implementation.6 12

A number of the original authors identified a range of successes or failures with respect to different clinical contexts. Implementation on critical care units was perceived as more successful than uptake in acute elderly care wards6 26 perhaps because health workers on critical care units are more aware of the importance of infection prevention.26 The need to address specific challenges such as differences in hand hygiene adherence between different locations9 and reducing MRSA acquisition19 23 were identified as motivators for change and being able to secure improvements in compliance.

The inability of infection prevention staff to undertake additional tasks related to the HHI was identified as a barrier to improved adherence,12 13 16 25 and there was recognition that successful hand hygiene initiatives require considerable commitment of resources.23 It was also noted that adherence varied with the particular daily demands placed on health workers in terms of staff availability and fluctuating patient case-mix.26

Discussion

We have taken a novel approach to evidence synthesis, adopting the example of interventions to improve adherence to hand hygiene protocols in patient care, we demonstrated that it is possible to apply the principles of evidence synthesis to interpretation and commentary included in epidemiological studies. Combining this approach with the findings of traditional systematic reviews would demonstrate whether the intervention is effective and how it exerts its effects and offer messages for sustainability and transferability to other contexts.

The thematic synthesis identified three major analytical themes relating to explanations of outcomes for the HHIs: methodological explanations for failure or success and two related themes that address issues associated with implementing HHIs: successful implementation needs leadership and cooperation from throughout the organisation and understanding the context and aligning the intervention with it drives implementation.

The first theme focused on aspects related to the internal validity of the studies, and methodological explanations mostly related to failure to demonstrate the effectiveness of HHIs. This analytic theme was evident in the majority of papers, reflecting findings from the Cochrane Review2 and other authors who have identified directions for future hand hygiene research.28 The Hawthorne effect was most frequently mentioned, followed by other sources of bias. Other reviewers have observed that although hand hygiene is frequently described as a simple preventative measure, HHIs are hard to design and conduct.29 Some authors, not meeting the eligibility criteria for our review, have employed the Hawthorne effect as part of the intervention.30 31 In these studies, it was deemed successful and is worth considering explicitly as part of an HHI.

Obtaining accurate and valid measurements of hand hygiene adherence is especially difficult in relation to the Hawthorne effect and as a result of observer error, failure to train observers, lack of inter-rater reliability and the challenge of documenting hand hygiene opportunities and events in busy clinical areas.32 Sustainability and methodological adaptations to achieve internal validity were identified by a number of the original authors. Central to success were HHIs that had sufficient follow-up to demonstrate sustainability and the implementation of techniques to ‘refresh the message’ in terms of feedback and performance benchmarking.

Theories of behavioural change were identified as helpful by a number of the original authors. It has already been suggested that theoretical frameworks from the behavioural sciences should be used to underpin HHIs,33 34 but these were employed in only a quarter of the studies. A different theory was applied in each, but all were thought to enhance understanding of hand hygiene behaviour and contribute to improved adherence. In one case, stakeholders suggested that the theoretical framework might help improve performance of other patient safety issues.9

A number of individual descriptive themes contained in two of the major analytic themes suggested challenges to implementation relating to institutional support and context. These concerns reflected a very broad spectrum of issues rather than a single barrier described in depth. Descriptive themes relating to implementation were less well developed than the themes relating to methodology, unsurprisingly given the focus on internal validity and study design that exist in evidence-based healthcare and the scant attention paid to issues of context and external validity.35 The importance of institutional support and context were novel findings not apparent in traditionally conducted systematic literature reviews of HHIs such as our Cochrane review,2 which focused on internal threats to validity.

Leadership from all levels of the organisation, especially from senior management and clinicians, was identified as a key to success. Visibility of senior staff ‘walking the walk’36 and ‘buy-in’37 were especially important. Stakeholder involvement was often lacking or unsuccessful however. Attempts to engage patients or the public were reported in only two studies, although international policymakers recommend including them in initiatives to prevent infection and reduce the risks of antimicrobial resistance.38 These attempts were viewed as problematic and identified as a barrier to implementing the HHI. None of the studies used a theory of leadership, despite identifying leadership as crucial. New studies would benefit from adopting a defined framework for leadership, such as using ‘Leadership and Organizational Change for Implementation’ (LOCI),39 which would enable the leadership components of an HHI to be theoretically driven and individually evaluated separately from bundled components.

Previous work suggests that contextual differences between organisations and clinical settings can affect the uptake of innovation and that initiatives successful in some settings are not always effective in others.40–42 As in the HHIs, these variations are attributed to differences in local culture, acceptability to staff and patients, patterns of work and changes in the same organisation over time. In many of the settings where the HHI took place, it would have been superimposed onto existing organisational and national policies. Infection prevention ‘fatigue’ may have undermined impact.

Study design did not appear to affect the type and detail of reporting. We reviewed randomised trials and rigorously undertaken ITS studies. The purpose of randomisation is to remove the effects of confounding variables on trial outcomes.43 A supposed advantage of ITS studies is the ability to take into account the impact of factors that might influence outcomes.44 We therefore anticipated that trials would contain less interpretation and commentary than ITS studies. Surprisingly, three of the most highly informative studies were randomised trials.6 8 9 Only two of the most highly informative publications were ITS studies.23 25 Of the nine ITS studies reviewed, six were not especially rich in interpretation or commentary.18 19 21 22 24

The aim of thematic synthesis was to understand the original authors’ interpretations and insights into what made an HHI successful or otherwise. Our approach to such ‘contextual data’ in reports of epidemiological studies is novel, and we consider that such an approach combined with traditional systematic reviews (including meta-analysis where possible) may provide additional insight. Our approach has provided new insight into reported factors influencing the success of HHIs. For example, authors of the primary studies placed great emphasis on need to improve approaches to implementation of the HHI, particularly in terms of engaging organisations holistically and leveraging leadership and implementing agile interventions sensitive to the local context and setting.

Advocates of evidence synthesis acknowledge that its effectiveness depends on the amount of information provided in primary studies.45 We confirmed this finding. Some publications contained particularly detailed interpretation and commentary,6 8 9 23 25 while others comparatively little.7 10 17 19 The value of thematic synthesis is also constrained by the type of information presented. What the original authors chose to write about and the amount of interpretation and commentary they included influenced our ability to synthesise and integrate the body of research as a whole. This limitation necessarily restricts the extent to which thematic analysis can be applied to data not collected in the usual way in qualitative enquiry. Nevertheless, there were sufficient data in most of the papers eligible for inclusion in thematic synthesis to conduct such analysis.

Messages to inform future research, policy and practice

Thematic synthesis identified key areas for research in relation to methodological rigour and implementation. The original authors expressed greatest concern in relation to the Hawthorne effect and other sources of bias, and indeed methodological shortcomings were sufficient to cause one research team12 to question the value of the HHI. Such misgivings may have been overemphasised, as HHIs are theoretically effective through breaking the chain of infection, and there is evidence from other studies that they can generate positive outcomes.30 46 Adaptations to demonstrate sustainability, including having a sufficient follow-up period, are required. Better controlled studies with improved hand hygiene monitoring would increase the credibility of the evidence supporting hand hygiene as the foremost infection prevention strategy. Our findings reiterate messages from Pittet’s seminal work in Geneva46: organisational support is central to the success of HHIs. Its importance is emphasised in WHO guidelines1 that also recommend customising HHIs to meet local needs. This requirement calls for greater understanding of how HHIs exert their effects at local level in response to specific needs and challenges and to enhance sustainability and transferability. Details of organisational support and context need to be clearly described in publications of HHIs so that a proper assessment of their external validity and applicability to other settings can be undertaken.

Leadership was identified as a key element of success, but the approach to leadership was neither defined nor theoretically underpinned in the studies. Adapted approaches such as LOCI39 would enable the role of leadership in the success of HHIs to be clarified and its role in the success or otherwise of HHIs to be evaluated separately. Such an approach may help to provide a clearer specification of HHIs especially where they are bundled, so that the role of different levels of leadership (eg, frontline managers or middle managers) and styles of leadership (eg, transformational or transactional) is made clear and their contribution to the success of the HHI is explicit. There is a wider scope for employing systematic approaches to studying the implementation of HHI interventions more widely, such as the Consolidated Framework for Implementation Research.40 Such an approach would help to guide systematic assessment of the hierarchical contexts that HHIs are situated within and help to systematically identify factors that might influence intervention implementation and effectiveness, potentially increasing the rigour of the research into HHIs and our ability to interpret the findings and generalise from them.

Finally feasibility studies are widely advocated to inform study design and methods, refine interventions, maximise acceptability to stakeholders and promote implementation.34 Many of the perceived methodological failings and implementation failures described above could have been avoided or reduced if more thorough preparatory work had been undertaken, accompanied by process evaluation. A quarter of the research teams reported feasibility studies, but in two cases, they did not prevent problems related to lack of acceptability and none adequately addressed the methodological challenges later identified by the original authors.

Study limitations

Systematic reviews of evidence synthesis cannot generate meaningful findings unless searches are rigorously undertaken and the included studies are robust.45 Our included publications met rigorous Cochrane requirements, but our second updated Cochrane review2 demonstrated that although they were the best available, certainty of the evidence was only moderate or low, with implications for validity. It is possible that more recent HHIs meeting the Cochrane criteria have now been published. They were not considered in this thematic synthesis. The value of thematic synthesis was further constrained by the amount and quality of interpretation and commentary included in the primary studies.

Conclusions

In this evidence synthesis, we identified three themes offering explanations of the original authors’ interpretations for the success or lack success of HHIs: methodological limitations affecting the internal validity of studies, implementation, external validity, organisational support and the need for HHIs to align with the existing context in the settings where implementation was attempted. New directions for research emerged: exploration of ways to avoid the Hawthorne effect; exploring the impact of individual components of bundled HHIs; the use of theoretical frameworks to underpin behaviour change and HHIs; the potential to embed the HHI into a wider patient safety and quality initiative; adaptations to demonstrate the sustainability of HHIs; and the development of a systematic approach to implementation. They need to be answered before policy and practice to increase hand hygiene adherence can advance.

References

Supplementary materials

Footnotes

  • Contributors The study was conceived by DJG and ND; all authors contributed to the analysis, DJG undertook initial drafting to which all authors contributed.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent for publication Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.