Background A multicentre, multinational research study requires careful planning and coordination to accomplish the aims of the study and to ensure systematic and rigorous examination of all project methods and data collected.
Objective The aim of this paper is to describe the approach we used during the HANDOVER Project to develop a multicentre, multinational research project for studying transitions of patient care while creating a community of practice for the researchers.
Method We highlight the process used to assure the quality of a multicentre qualitative study and to create a codebook for data analysis as examples of attending to the community of practice while conducting rigorous qualitative research.
Findings Essential elements for the success of this multinational, multilanguage research project included recruiting a strong research team, explicit planning for decision-making processes to be used throughout the project, acknowledging the differences among the study settings and planning the protocols to capitalise upon those differences.
Conclusions Although not commonly discussed in reports of large research projects, there is an underlying, concurrent stream of activities to develop a cohesive team that trusts and respects one another's skills and that engage independent researchers in a group process that contributes to achieving study goals. We discuss other lessons learned and offer recommendations for other teams planning multicentre research.
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/3.0/ and http://creativecommons.org/licenses/by-nc/3.0/legalcode
Statistics from Altmetric.com
The European Union FP7 Health research Programme commissioned the European HANDOVER Project in 2008, a 3-year, 3.5 million euro programme designed to examine transitions of patient care from the acute hospital to the primary care setting.1 ,2 Six European countries—Italy, the Netherlands, Poland, UK, Spain and Sweden—participated in this study (see figure 1).
The study design required a mixed-methods approach in which researchers conducted semistructured interviews with key stakeholders (patients and their care providers) and focus groups with patients, patient representatives, and hospital-based and community-based providers (physicians and nurses) to gain insights into handover practices at the hospital to primary care interface (ie, referral and hospital discharge). While the overall focus was on the hospital to primary care interface, each country had a clinical focus that represented the interest areas of the researchers: general medicine patients in the Netherlands, under-represented minority patients in Spain, emergency department patients in Italy and Sweden, and geriatric patients in Poland. Our research plan extended beyond the traditional set of qualitative methods to develop and apply quality improvement tools (specifically Ishikawa diagrams, process mapping, narrative analyses of near misses and patient stories, and review of artefacts) to enhance participants’ understanding of the complexity of patient handover processes as well as explore facilitators and barriers to effective handovers.3 This innovative approach was needed to address the increasingly acknowledged implementation gap and these tools allowed us to build bridges between qualitative research and implementation research as we began to translate research findings into better handover practice. We also used a series of quantitative methods to assess the cost effectiveness of handover training.4
From the initial funding of the HANDOVER Project, the Project's leadership agreed that, in addition to the research activities to meet the investigational aims of the project, there would be a separate set of activities to develop a community for the researchers. In essence we created a ‘community of practice’ which we named the European Handover Research Consortium. It was designed to assist the researchers involved in the Project define a common purpose, build research teams and engage researchers further in the larger set of research topics, while engendering trust among the team members. We found many examples and guidelines in the literature about developing multicentre trials with the focus on developing, reporting and managing the research.5–8 In contrast, we have found very little reported in the literature about how others have done what we set out to do—create and support a multicentre research team and enhance the quality and value of the work completed under the overall project through rigorous qualitative research.9–11 While everyone would agree that the primary focus of a research project is to conduct the research, based on our own experiences as researchers and as members of research teams, we felt that the best outcomes would come from an explicit focus on developing and supporting the researchers.
In this paper we describe the process we used to create a community of practice for the HANDOVER Project, address the challenges of implementing a complex multisite project on patient transitions, identify strategies used to manage the project, and provide recommendations for teams planning multisite research.
Creating a multicentre community of practice
A community of practice is ‘a group of people who share a concern, a set of problems or a passion about a particular topic, and who deepen their understanding and knowledge of this area by interacting on an ongoing basis.’12 Communities of practice are characterised by the domain (an identity defined by shared interest, commitment and shared competence), the community (joint activities and discussions to help members of the community and to share information), and the practice (the shared repertoire of resources, experiences, stories and tools). The combination of these three elements—as well as the development of these elements in parallel—creates the community of practice.13
According to Wenger and colleagues, a community of practice can be distinguished from formal departments and project teams along the following five dimensions12:
Purpose: to create, expand and exchange knowledge, and to develop individual capabilities;
Membership: self-selection based on expertise or passion for the topic;
Boundaries: Communities of practice have fuzzy boundaries (in contrast to a business or organisation with distinct boundaries);
What holds them together: passion, commitment and identification with the group and its expertise; and,
Life cycle: communities of practice evolve and end organically; they last as long as there is relevance to the topic and interest in learning together.
One could argue that a research project would not naturally fit this definition, yet some authors suggest that community of practice actually is an umbrella term for a number of different organisational groupings that are characterised by the support for formal and informal interaction between novices and experts, the emphasis on learning and sharing knowledge, and the investment to foster a sense of belonging among members.14 We chose to use the term for our study because we wanted to focus attention on the stream of activities that are required for building the research team, while simultaneously and rigorously conducting the research that the team received funding to achieve. Although not often discussed and often taken for granted, the functioning of the research team directly contributes to, and influences the success of the research project.
Wenger suggests seven principles for cultivating a community of practice: (1) design for evolution, (2) open a dialogue between inside and outside perspectives, (3) invite different levels of participation, (4) develop both public and private community spaces, (5) focus on value, (6) combine familiarity and excitement, and, (7) create a rhythm for the community.12
For the purpose of the HANDOVER Project, the community of practice included the group of researchers from the six organisations, as well as patient advocates, that came together to conduct research into transitions of patient care. Each of the participating countries conducted research at the local level and built local alliances with healthcare providers and patients. The opportunity to work together as a group occurred through quarterly face-to-face meetings rotated among the study sites, site visits, monthly conference calls and frequent electronic communications. The interactions were structured—responsibility for hosting the face-to-face meetings was shared by the participating countries, meetings included working sessions as well as knowledge transfer sessions, serious games were used to build team cohesiveness,15 and group members became comfortable with the rituals that evolved over the life of the project. The project increased the appreciation for research by holding training seminars for the researchers in quantitative and qualitative methods, allowed an opportunity for scholarly exchange, and accented the positive use of resources in all settings.
Because building the community of practice occurred in conjunction with planning and conducting the research, we were able to use this as an opportunity to co-create the science and methods. For example, as the project began, we faced the daunting task of ensuring consistency of data collection and analysis across multiple sites in five countries and using five different languages. The research objective—to identify the barriers and facilitators to effective handovers in the transition of patients between the hospital and community setting—required a robust quality assurance plan to systematically examine all project methods, data collected and documentation. As described in the following paragraphs, developing a quality assurance and creating a codebook for data analysis are two specific activities that illustrate how we attended to the emerging community of practice while launching the research project.
Developing a quality assurance plan
Quality assurance in a research project is the systematic and independent examination of all project methods and documentation. A quality assurance plan should ensure that researchers carefully document the methods and protocols and provide consistent methods across participating sites (and in the case of multinational research, in different languages). Quality assurance of a research project is a continuous, dynamic process, particularly for qualitative studies and research in emerging areas. Although the European HANDOVER Research Consortium established methods and protocols at the beginning of the project, we found that the expectations evolved and new requirements emerged as the researchers became more familiar with the territory under exploration and the organisational relationships that needed to be carefully navigated. The emerging nature of these dynamics is at the heart of complex social group interactions.16 Our aim was to monitor the research as it unfolded and evolved throughout the life of the project, maintain methodological rigour and ensure consistency of the research process across multiple interventions across five project sites in five different countries.
To meet this aim and address the evolving risks, we created an approach to assess and report on the investigative work conducted under the aegis of the project, monitor risks and prevent ‘drift’ of the project's scope. We drafted criteria for reporting qualitative research findings from two respected available sources.17 ,18 The criteria were presented to the HANDOVER research team during face-to-face project meetings. The research team reached consensus through a facilitated discussion, about which elements were most relevant and would be part of a regular audit function of the HANDOVER Project. The criteria were used to assess the work across the research sites to create a template for the HANDOVER Quality Assurance Report. Table 1 outlines the criteria and illustrates how we operationalised defining these ambiguous concepts to make them clearly distinguishable or measurable by the HANDOVER researcher team19 (the completed quality assurance report is available on request).
Each research site agreed to complete the reporting template for each phase of data collection (ie, after interviews, after focus groups, after process mapping, after artefact analysis). Applying the tool in a risk mitigation approach is innovative—however adhering to a quality assurance plan is not a new concept—but the process that we used to adapt the tool and build consensus by engaging those who would be expected to comply with the criteria was an important step in its implementation20 and in creating the HANDOVER community of practice.
Creating a codebook for data analysis
Although the main focus of the quality assurance report was the data collection phases, our overall quality assurance approach included an extensive data analysis plan with a consensus approach to the analysis process to maximise intercoder reliability in coding open-ended data. Coding is the interpretative process in which conceptual labels are given to the data.21 Codes are words or devices for identifying themes. For the HANDOVER Project, the process involved segmenting the data, developing a draft codebook, testing the reliability of the sample of data by two independent coders, checking and modifying the codebook before final coding and finally, applying a quality control process of all data. Our task was to enable and support the researchers in immersing themselves in the data, yet provide enough structure to the emerging codes to allow the analysis to make valid comparisons across sites during the analysis and interpretation of the results.
While each researcher collected data in their local settings using their native language, we required a common language for coding and analysis (just as we needed a common language for our face-to-face meetings) to facilitate the sharing of information across sites and to disseminate our research findings to wider audiences. We agreed to develop codes for data analysis in English so that they could be shared by the project group, however, we agreed that the language fidelity and fluency of the healthcare providers in the respective countries required that the analysis be conducted in the original language of the interviews by local research teams.
Researchers in each country generated a list of narrative codes as they completed the stakeholder interviews and focus groups. The generated codes were circulated between researchers in each of the countries. The list of codes was developed into a shared codebook, during face-to-face meetings, conference calls and electronic mail correspondence. Our first attempt at creating a consensus codebook was at a face-to-face meeting in September 2009 hosted by our colleagues at the Avedis Donabedian Institute in Barcelona. Researchers came to the meeting armed with the codes that had emerged from the locally gathered data. To achieve consensus, we engaged them in an exercise to create an affinity diagram where we separately recorded each code, similar codes were assembled into groups, duplicate codes discarded and groups of similar codes assigned a descriptive label.22 Regular monthly conference calls and two face-to-face visits were subsequently held to refine the codebook as other codes arose during the analyses, and to group additional codes that were related to the same phenomenon into unique categories. The process resulted in a consensus codebook of 84 codes with nine categories. We achieved agreement about the meaning of the English translation of the developed codes before progressing the analysis in the native language of the participating countries. The photo montage in box 1 highlights the experiential nature of this exercise that was used to engage the researchers physically as well as intellectually.
The quality assurance process was a prerequisite for ensuring reliable and valid analyses across our large-scale, multilanguage, multinational project. Our aim in creating a quality assurance reporting template was to enhance the internal and external reliability of the research process—from data collection to analysis—so that the participating researchers, as well as the larger research community, would have confidence in the results of the study. These efforts allowed us to standardise the methodological approaches across the settings and made it possible to replicate the study in other settings. We also found that the process of involving the team in creating the quality assurance reporting tool created buy-in among our researchers and contributed to the community of practice.
Consideration of the research methods
Health sciences research that primarily deploys quantitative methods, such as clinical trials, has clear, well-recognised approaches to assure the quality and reliability of research methods and protocols, including identifying responsibilities of key personnel, and adhering to a timetable.6 Some have questioned the rigour in assuring reliability of qualitative research.19 In the HANDOVER Project, combining a qualitative approach with traditional quality improvement methods, allowed us to gain insight from participants and contextualise our findings on how best to overcome the barriers to improving patient handovers. Quality improvement work is inherently ‘qualitative’ and can benefit from an understanding of the use of qualitative research methodologies. At the same time, it is essential that those who successfully manage an improvement process understand the context of the clinical setting while testing changes that lead to improved outcomes.
There are different challenges and threats to qualitative studies, compared to quantitative studies. In quantitative research, multicentre randomised studies are held as the gold standard because they offer a more robust data set, allowing the detection and confirmation of smaller effect indices, and offering added evidence around the generalisability of interventions. In contrast, multicentre studies present a particular methodological challenge in qualitative research as local context (which makes interventions more ‘real’) adds complexity and opportunities for introducing confounding variables. Generalisability—extending the research findings from the study population to the population at large—is not possible given that the size of the sample (generally a small group of people or a small number of cases) and the context is generally not representative of the larger population. However, interventions may be applicable to other distinct populations, and moderatum generalisation—in which investigators, based on what they learned, transfer and apply the findings to other settings—may be appropriate.23 ,24
In multicentre and multinational research, geographic distance between the researchers adds complexity to the day-to-day research activities. Particularly, daily work cannot be observed on an ongoing basis by study collaborators. Ensuring the ongoing alignment of the researchers is essential in maintaining consistency of the ongoing research process.19
An added challenge of a multinational study is addressing language barriers (in the case of the HANDOVER Project, five different languages) and differences in nations’ health systems and cultural context that affects the research undertaken. The quality assurance plan helped us overcome this challenge by acknowledging the need to conduct research in the local language and agreeing on a common language for interpretation of results and preparation of manuscripts for peer review publication.
Lessons learned and recommendations
Success of the HANDOVER Project was facilitated by building the project on pre-existing relationships among the site investigators who then assembled a team including faculty, research coordinators, patients and PhD research candidates. Study sites offered a range of organisational models and expertise, and the project was designed to capitalise on the organisational and resource strengths of each setting.
While we were not explicit about designing the HANDOVER Project as a community of practice, it became clear during the study that introducing principles from the study of communities of practice into multicentre, multinational research projects allowed us to enhance the capacity for creating a research collaborative while consistently delivering high value products. We propose that other research projects should employ a parallel process to assess the functioning, joy and vitality of the research teams25 that co-create the science, the methods, and the research.
Given the barriers created by distance between study sites, time zones and busy work schedules, we cannot overemphasise the value of frequent face-to-face meetings in creating a shared understanding, support and mutual comfort despite the relative ease, and cost savings, of relying only on electronic and virtual meetings. Although technology and cost containment efforts support a move toward virtual meetings, where participants attend a meeting without leaving their local settings, face-to-face meetings, supplemented with frequent teleconferences, proved to be an invaluable component of building social capital and engendering trust among the project team. This procedure also stimulated and ensured a timely delivery of project milestones and deliverables.
Researchers are increasingly encouraged to establish international collaborations and undertake cross-national comparative studies. While there are many benefits of multisite, multinational research, such as sharing of expertise and resources and minimising duplication of studies, there are also many challenges and risks. Project management is complex, and requires a focus on risks, dependencies, and communication (intra-team, inter-team and up to management). Communication challenges are compounded by multiple languages and require active interventions to mitigate them, not just documentation in written plans. Explicit planning for decision-making processes to be used throughout the project, acknowledging the differences among the study settings and ensuring that protocols capitalise on these differences, and recruiting a strong research team were essential elements for success.
Finally, while much effort, justifiably, is focused on accomplishing the research, we feel that there is a need for a parallel approach that is ‘researcher focused’ to ensure that the project evolves in a supportive environment that allows the research team to conduct rigorous, high quality investigative work.
Collaborators The European HANDOVER Research Collaborative consists of: Venneri F, Molisso A (Azienda Sanitaria Firenze, Italy), Albolino S, Toccafondi G (Clinical Risk Management and Patient Safety Center, Tuscany Region, Italy), Gademan P, Göbel B, Kalkman C, Pijnenborg L (Patient Safety Center, University Medical Center Utrecht, Utrecht, The Netherlands), Wollersheim H, Hesselink G, Schoonhoven L, Zegers M (Scientific Institute for Quality of Healthcare, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands), Boshuizen E, Drachsler H, Kicken W, van der Klink M, Stoyanov S (Centre for Learning Sciences and Technologies, Open University, Heerlen, the Netherlands), Kutryba B, Dudzik-Urbaniak E, Kalinowski M, Kutaj-Wasikowska H (National Center for Quality Assessment in Health Care, Krakow, Poland), Suñol R, Groene O, Orrego C (Avedis Donabedian Institute, Universidad Autónoma de Barcelona, Barcelona, Spain), Öhlén G, Airosa F, Bergenbrant S, Flink M, Hansagi H, Olsson M (Karolinska University Hospital, Stockholm, Sweden), Lilford R, Chen Y-F, Novielli N, Manaseki-Holland S (University of Birmingham, Birmingham, United Kingdom).
Contributors Study conception and design: JKJ, MVD, PB. Acquisition of data: JKJ, MVD, PB. Data analysis and interpretation: JKJ, MVD, PB. Drafting and/or revising the article: JKJ, MVD, PB.
Funding We wish to acknowledge funding from the European Union, the 7th Framework Programme of the European Commission (FP7-HEALTH-F2-2008-223409). The study sponsor had no role in the study design, collection, analysis, interpretation of the data, or any other contribution.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data and tools are available upon request from the corresponding author.
Open Access This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.