Article Text
PDF
Abstract
Introduction The objectives of the Breakthrough Series Collaborative are to close the gap between what we know and what we do, and to contribute to continuous quality improvement (CQI) of healthcare through collaborative learning. The improvement efforts are guided by a systematic approach, combining professional and improvement knowledge.
Objectives To explore what the improvement teams have learnt from participating in the collaborative and from dealing with promoting and inhibiting factors encountered.
Method Qualitative interviews with 19 team members were conducted in four focus groups, using the Critical Incident Technique. A critical incident is one that makes significant contributions, either positively or negatively, to an activity.
Results The elements of a culture of improvement are revealed by the critical incidents, and reflect the eight domains of knowledge, as a product of collaborative learning. The improvement knowledge and skills of individuals are important elements, but not enough to achieve sustainable changes. 90% of the material reflects the need for a system of CQI to solve the problems that organisations experience in trying to make lasting improvements.
Conclusion A pattern of three success factors for CQI emerges: (1) continuous and reliable information, including measurement, about best and current practice; (2) engagement of everybody in all phases of the improvement work: the patient and family, the leadership, the professional environment and the staff; and (3) an infrastructure based on improvement knowledge, with multidisciplinary teams, available coaching, learning systems and sustainability systems.
- continuous quality improvement
- patient safety
- breakthrough collaboratives
- infrastructure of improvement
- critical incident technique, collaborative
- leadership
- organisation
Statistics from Altmetric.com
- continuous quality improvement
- patient safety
- breakthrough collaboratives
- infrastructure of improvement
- critical incident technique, collaborative
- leadership
- organisation
Breakthrough Series Collaboratives (BTSC) aim to close the gap between best and current practice, and to contribute to continuous improvement of healthcare through collaborative learning.1 The participants learn to combine professional and improvement knowledge, based on the understanding that healthcare is a complex system, and that professional knowledge alone is not enough to change the system when needed. Improvement knowledge is based on the theory that both the different elements (units, microsystems and individuals) in a system and their interactions constitute the system. To improve a system, we need to understand the context and pattern of the interactions, and their psychology, variation and epistemology. The model for improvement includes four elements that lead to progress: (1) measurements related to (2) aims and (3) ideas carried out in a Plan–Do–Study–Act cycle with (4) small tests of change. The cycle illustrates that improvement should be a continuous, timely and scientifically grounded process.2–5
The Norwegian Medical Association organised and funded four national BTSCs from 2002 to 2006 (table 1). A BTSC is a 6- to 9-month action-learning programme that brings together 20 to 30 clinical teams from different organisations, in our case hospitals, to seek quality improvement (QI) on a given topic. Each improvement team sends two to four representatives to the three learning sessions of the BTSC. The learning strategy combines theory and action.3 Subject matter experts demonstrate the quality gaps, and improvement advisors follow-up on the improvement process. Each advisor usually coaches two teams using email, telephone calls and visits.
Number of teams responding
Healthcare leaders need more evidence to know how to bridge the quality chasms and make care safer.6 International studies on the impact of quality collaboratives on healthcare show that further knowledge on BTSC efficacy, cost-effectiveness and success factors is needed.7–11 The focus on collaboratives as a whole may have resulted in a limited understanding of the experiences of individual teams in the collaborative process.5
There have been no previous studies of Norwegian BTSCs. Our goals are to explore to what degree the BTSCs are building a culture of continuous improvement, and identify the success factors of continuous quality improvement (CQI) hidden in the experiences of the improvement teams.
Method
The sample frame consisted of 101 improvement teams from the four BTSCs. The Norwegian Medical Association had saved the email addresses of the improvement team leaders, and some of the team members (N=121). This list was used to invite participants to four different focus-group meetings, covering Norway's five geographic regions. The number of improvement team members actually receiving the invitation is unknown due to transitions in the Norwegian healthcare system leading to turnover of staff and new email addresses. Twenty-one people responded positively to the invitation, and 19 physicians, nurses and psychologists actually participated in the focus-group meetings, representing 10–30% of the teams from each region.
Data collection
Critical incident data were collected in focus groups conducted by a research team of four experienced improvement advisors: one physician, two nurses and one bioengineer. The Critical Incident Technique (CIT) was used as the data-collection method.12 13 A critical incident is one that makes a significant contribution, either positively or negatively, to an activity.13
When seated around the table, the respondents were asked: ‘Think about the time when you were participating in the BTSC and the time after the project ended. What did you experience as promoting and/or inhibiting elements during and after the BTSC?’ The first focus group also wanted to discuss positive spin-offs from their BTSCs. The subsequent groups were thus asked to do the same. The story-telling moved around the table several times with minimal interruption by the data collectors. Their task was to support the process using active listening techniques, confirming what the respondents meant without asking other questions or providing explanations.13 14
The data collectors summarised the critical incidents on flipcharts in dialogue with the respondents who assessed the accuracy of the recorded information. The focus groups lasted about 45 min, and each respondent reported from five to 20 critical incidents and positive spin offs (comments) with a mean of 12 per respondent and a total of 233 comments. All comments were used and translated into English by a Canadian–Norwegian journalist.
Data analysis
The comments were analysed using three different approaches. First, the different domains of knowledge reflected by the comments were explored using a theoretical framework — ‘The eight domains of knowledge.’15 Prior to this, the operational definitions of the domains were updated (box 1) based on published sources.15 16 Comments covering more than one domain were divided into subparts, increasing the total number of comments from 233 to 262. Next, the comments were sorted into the eight domains by the research team using a consensus approach, and examples were chosen to illustrate each domain.
Operational definitions of the eight domains of knowledge
D1. Healthcare as a process, system‘The interdependent people (patients, families, eligible populations, care givers), procedures, activities and technologies of healthcare-giving that come together to meet the needs of individuals and communities, and the activities in which that occurs.’ [A] The understanding of and the ability to ‘Improve connections and coordination with other microsystems.’ ‘Improve systems within which microsystems are embedded to keep all microsystems coordinated and focused on creating and testing changes around a common objective.’ ‘Invest in infrastructure to allow multiple clinical microsystems to design a coordinated journey for patients.’ [B]
D2. Variation and measurement‘The method and the use of measurement to understand the variation across and within systems to improve the design and redesign of healthcare.’ [A] The interest of measurement among the providers, and the availability of the recorded data. [C]
D3. Customer/Beneficiary knowledge‘Identification of the person, persons, or groups of persons for whom healthcare is provided or may be provided in the future, an understanding of their needs & preferences, the roles they may play in their own healthcare and the relationship of healthcare to their needs and preferences.’[A] The involvement of patients in the improvement efforts. [B and C]
D4. Leading, following and making changes in healthcare‘The methods and skills for designing and testing change in complex organisational care-giving arrangements, including the general and strategic management of people and the healthcare work they do in organisations.’ [A] The infrastructure of improvement, and the building of pride and confidence of improvement activities. [C]
D5. Collaboration‘The knowledge, methods and skills needed to work effectively in groups, to understand and value the perspectives and responsibilities of others, and the capacity to foster the same in others, including an understanding of the implications of such work.’ [A] The internal group process of the improvement team, and the external group process (the ability to collaborate with the rest of the unit). [C]
D6. Social context and accountabilityAn understanding of the social contexts (local, regional, national, global) of healthcare-giving and the way that expectations arising from those social contexts are made explicit. This includes an understanding of the financial impact and costs of healthcare. [A]
D7. Developing locally new useful knowledge‘The recognition of the need for new knowledge in personal daily health professional practice and the skill to develop new knowledge through empiric testing and reflection on daily work experiences.’ [A]
D8. Professional subject matter‘The health professional knowledge appropriate for a specific discipline and the ability to apply and connect it to all of the above. Often summarised in core competencies published by professional boards, accreditors, and other certifying entities.’[A]
References: [A] Batalden and Splaine.15 [B] Nelson et al.16 [C] The authors of this paper.
The second approach explored the learning potential contained in the experiences of the participants using the theoretical framework of the BTSC—the Continuous Improvement of Health Care.3 Upon reflecting on the material, a pattern of success factors for a CQI system emerged. The selection of comments to illustrate the success factors were discussed and agreed upon by the research team.
Third, the 262 incidents were reanalysed, with a focus on understanding to what extent the critical incidents reflected the importance of the system as compared with that of the individual in making improvements happen. With this in mind, each of the data collectors coded the comments from their own focus group, and the comments of another group. The differences between the two versions of findings were discussed, reaching consensus within the research team.
The CIT method illuminates the complexity of change stimulated by collaboratives; it reveals a mix of factors associated with both collaboratives and the local contexts that are the focus of change. To make it easier for leaders to develop a CQI system based on this study, we used our expertise as improvement advisors to sort out facilitating factors mainly related to the collaborative learning process (marked in table 2A–C), from comments mainly related to the organisational system of the respondents.
Success factors of a continual improvement system
Results
The elements of a culture of improvement are revealed by the comments, and reflect the eight domains of knowledge, as a product of collaborative learning. A selection of comments sorted into the eight domains is presented in appendix A.
The respondents provided about 80 examples of positive spin-offs from their BTSCs. Benefits included: (1) the spread of improvement efforts and methods to other areas and sites through projects, conferences and papers; (2) a focus on improvement and quality made the staff proud of their organisation; (3) patients are better informed and more engaged in planning the care than before; (4) practical improvements have been made that impact daily routines (eg, forms and checklists), making it easier to work smarter; and (5) improved professional knowledge and reputation after the BTSC make it easier to recruit the best personnel.
Nearly 10% of the 262 comments reflect the importance of the effort of individuals, and about 90% of the comments in each region provide information about the content of what we define as a CQI system (table 3).
Categories of comments and their regional distribution
The comments produce a pattern of three success factors and 12 success criteria for a CQI system displayed in box 2. Tables 2A–C provides detailed information about the system, illustrated by the comments written in italics. Forty-four per cent of the comments are mainly related to the collaborative system, labelled collaborative learning (CL). The remaining comments are mainly related to features of the participating organisations.
Continual improvement system
Success factor I: information
1. Provide continual and reliable information about best practice
2. Provide continual and reliable information about current practice
3. Benchmark systems and outcomes to others
Success factor II: engagement
4. Anchor the improvement work to the leadership at all stages
5. Focus on and engage the patient and family in all stages of the improvement work
6. Anchor the changes to the professional environment
7. Engage the staff in all stages of the improvement work
Success factor III: infrastructure
8. Base the infrastructure on improvement knowledge
9. Multidisciplinary improvement teams tailored to the topic
10. Develop a learning system tailored to the different target groups
11. Develop a system to facilitate the improvement work
12. Develop a follow-up system to secure sustainability
Discussion
Method
The research team studied improvement initiatives in different organisations to generate concepts and theories of CQI based on the CIT. In a comprehensive review of CIT studies, Gremler indicates that this method is suited to capture the perception of individuals from different cultures because it invites respondents to share their point of view, rather than respond to researcher-initiated questions. CIT is an inductive method that needs no hypothesis; patterns are formed as they emerge from the responses, allowing the researcher to generate concepts and theories, observing cognitive, affective and behaviour elements to understand the situations the respondents are describing.13 Based on the richness of information provided by our 19 respondents, we found the CIT to be a useful method for gathering this kind of data.
Interpretation
The comments in table 2A–C indicate that the many positive spin-offs reported by respondents are due to the effectiveness of the breakthrough series model (labelled with [CL]) as well as features of the participating organisations (unlabelled comments). Some partly CL-related comments are not labelled because of their high degree of context dependency: interest in measurement, influence, credibility, early adopters, balanced measurement and patient involvement. This indicates the complexity and inter-relationship of the data.
The collaborative provided both professional and improvement knowledge to the organisation in a system of mutual learning and attention, offering an academic approach to improvement that promoted the participants' belief that there was something in it for them. It may be difficult for an organisation to achieve these benefits without the pace and structure of a BTSC.
The main power of the BTSC model is probably its underpinning of professional knowledge. Because this is such a major part of the collaborative, it may be easily taken for granted and was not reported as a big issue by the respondents. This, however, should not lead one to think that CQI efforts should ignore the importance of combining generalisable scientific evidence with improvement knowledge to make changes happen in a particular context.3 17–19
A lack of follow-up by an organisation's leaders seems to threaten sustainability. To make CQI efforts become a part of daily work, it has to be on the agenda of daily practice and ordinary staff meetings. This will require fundamental changes in the meeting structure of most healthcare institutions.
Relation to Other Evidence
Our findings support the existing theory of CQI and organisational change,2–7 15–32 including the role of measurements.33–38 We were somewhat surprised how well the comments of the 19 teams covered the eight domains of knowledge (appendix A). This supports the well-established learning principle of combining theory with practice,1 3 7 20 and the generalisability of the existing theory of improvement knowledge, reflecting patterns of common experiences.
A PubMed review using the MeSH term ‘breakthrough collaborative’ produced 61 relevant articles. Fifty-seven of these focused on professional subject matters, such as describing to what extent the collaborative had been able to close the gap between best and current practice. Seven provided information on facilitators,39–45 while only four articles covered the eight knowledge domains aiming ‘to develop a shared baseline for understanding clinical improvement science.’6 46–48 Our findings are generally consistent with the 18 facilitators/barriers discussed in these papers. They described 14 facilitators related to infrastructure for improvement, three related to engagement and one on information. The most frequent facilitators mentioned are strong senior leadership and support, multidisciplinary teamwork skills, and ongoing monitoring of the process targeted for improvement. We found no studies of facilitators and barriers encountered later than 1 year after a BTSC.
Our analyses revealed that 90% of the critical incidents reflected the need for a system of CQI. However, only two of the 19 clinicians in our focus groups addressed the facilitators and barriers to their organisational quality system, and only one of the 61 articles did so.6
The need for an improvement infrastructure has been confirmed by others.5 8 20 30 31 49–51 One might ask, ‘Is there a firewall between top-down and bottom-up approaches to QI?’ Studies suggest that clinicians often think that quality is their personal responsibility, and do not take into account the complexity and contributions of the system.18 52 When clinicians become leaders, will their strategy be coloured by this thinking? The targets of the Norwegian national QI strategy are aligned with CQI theory, and go well with the ‘bottom up’ specifications contained in tables 2a–c and appendix A.53 The MARQuIS study on QI strategies in eight European countries does not answer the question because of substantial international and intranational variations. Only one of the specifications used to assess the application of QI strategies (patient-centredness) goes well with the ‘bottom up’ approach of our material.54 Other parts of the MARQuIS study found that 44% of the hospitals provided QI training and support, and 56% had systems for monitoring professional performance, but only 9% had information systems to provide data on the quality of care.50 The implementation of QI strategies in Europe may sometimes be remote from routine clinical practice.51
Limitations
This study has several limitations.
Representativity
The representativeness of the teams was low (19%), especially among the first collaboratives. Although two of the regions did not reach the ideal focus group size of five to eight respondents, their comments were consistent with those from the other groups (table 3). Qualitative studies often use small samples. In general, few new perspectives emerge by extending the number of respondents beyond 15–24.55
Sampling issues
The findings are based on the perspectives of improvement team leaders who volunteered to participate in focus groups. The amount of positive spin-offs, and the fact that most respondents had documented improvements in their final reports, shows that we were most probably informed by winners. Their viewpoints may not fully represent the whole spectrum of participants, especially the viewpoints of the less successful efforts. Consequently, the inhibiting elements encountered by the teams may not have been fully revealed.
Second, the sampled BTSC are from the psychiatric sector of healthcare, and one may conclude that the results are limited to this field. However, the matching theoretical framework, and the reported facilitators in 10 other BTSC articles from the non-psychiatric settings, does not support this.
Sources of bias
It would have been better to have had an independent facilitator lead the CIT focus groups and analyse the data because some of the authors served as improvement advisors in the BTSC. Our connection to the collaboratives could prompt us to glorify the outcomes. Second, the fact that we are experts on CQI may also lead us to see patterns that the respondents would question, especially about the need for a system of CQI. With this source of bias in mind, we analysed the comments of each other’s focus groups. The result did not confirm any such bias. Third, the CIT method has been criticised as being flawed by recall bias.13 To avoid this, the notes on the flipchart were made in dialogue with the respondents in the data-gathering phase. Still, we cannot know for sure that they were really assessing the accuracy of the recorded notes. We may also have recalled their meaning in the light of our theoretical knowledge.
Conclusion
Emerging elements of a culture of CQI were identified by the former participants of BTSCs. Their individual improvement knowledge is not enough, but their collective knowledge, systematised by the research team, and based upon theory, has generated three success factors of a system of CQI.
By listening to the voice of the patients when ‘travelling’ through healthcare, by listening to the voice of the process when monitoring variation and change, and by listening to the voice of science, the improvement gaps will most probably be revealed.
By involving the patient and family, by anchoring the efforts to the leadership and the professional environment, and by utilising the power of the personal ambitions of the people involved, we have an improvement strategy. If everyone has two jobs—making and improving healthcare—we develop an improvement culture.
Based on these factors, and an available infrastructure to facilitate improvement efforts, we generate a system of continuous improvement.
Future endeavours to make sustainable improvements in complex healthcare systems without the support of a system of continuous improvement are not likely to be successful.
Acknowledgments
The authors thank I-C Helljesen, for her participation in the translation process, and H Asbjørn Holm, for his help and support with this study.
References
Footnotes
ASB is the research team leader, collected the data from the focus groups and had the main responsibility for the study and the manuscript, the database and its content. AS assisted with the data collection and organising of the database. She participated in data analysis and the translation process, and in the writing of the manuscript. PH supervised the study, the research process and the writing of the manuscript. GSH assisted in planning and organising the focus-group meetings, and participated in the data analysis, translation process and final preparation of the manuscript. BN collected data from one of the focus groups, participated in the data-analysis process, has an intimate knowledge of the database and participated in writing the manuscript. ECN assisted in planning the analysis of the qualitative information, outlining the contents of the article, and editing and reading the manuscript.
Funding The Norwegian Medical Association Quality Assurance Fund provided partial funding for this study.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.