Article Text

PDF

Progress of unit based quality improvement: an evaluation of a support strategy
  1. L Wallin1,2,
  2. A-M Boström3,
  3. G Harvey4,
  4. K Wikblad5,
  5. U Ewald1
  1. 1Department of Women’s & Children’s Health, Uppsala University, Uppsala, Sweden
  2. 2Public Health & Caring Sciences, Uppsala University, Uppsala, Sweden and Dalarna University, Falun, Sweden
  3. 3Department of Nursing, Karolinska Institute, Stockholm, Sweden
  4. 4RCN Institute, Oxford, UK
  5. 5Medicine and Care, Linköping University, Linköping, Sweden
  1. Correspondence to:
    L Wallin, Dalarna University, Campus Falun, S-791 88 Falun, Sweden;
    lwa{at}du.se

Abstract

Objective: To evaluate a strategy for supporting nurses to work with quality improvement (QI).

Design: Post-intervention evaluation.

Study participants and intervention: 240 nurses participated in a uniformly designed 4 day basic training course in applying a model for QI. Of these, 156 nurses from over 50 healthcare institutions constituted the generic education (GE) group while 84 nurses from 42 neonatal units took part in a project to develop national guidelines, constituting the targeted intervention (TI) group.

Method: Postal questionnaire 4 years after the training courses.

Results: The response rate was 80% in the TI group and 64% in the GE group. Nurses in the TI group had a significantly higher rate in completing all phases of the QI cycle (p=0.0002). With no differences between the groups, 39% of all nurses were still involved in QI work 4 years after the training courses. Three factors were significantly related to nurses continuing their involvement in QI projects: remaining employed on the same unit (OR 11.3), taking courses in nursing science (OR 4.1), and maintenance of the same QI model (OR 3.1). Reported motives for remaining active in QI work were the enhancement of knowledge, influence over clinical practice, and development as a nurse. Reasons for discontinuation were organisational restructuring, a lack of facilitation and knowledge, and change of workplace.

Conclusions: Participation in a national guideline project, including a common focus for improvement, facilitation and opportunities for networking, seems to have enhanced the ability to carry out the process of QI, but not to sustain the QI work over a longer period.

  • quality improvement
  • nursing
  • clinical audit

Statistics from Altmetric.com

BACKGROUND

Published studies on the outcomes and effectiveness of quality improvement (QI) vary from severe doubt to overflowing enthusiasm. Much of this literature is anecdotal, describing what was done and what seemed to have been accomplished, often in a specific location and over a short period,1 but a trend towards controlled studies has recently been observed. Improved patient outcomes—as well as no changes or only small effects—are reported, although weak study designs often temper positive findings of QI programmes.2–,5 In parts of Europe QI is often described at an operational level as clinical audit, defined as a systematic and critical analysis of the quality of clinical care.6 The benefits of audit include improved communications skills, improved patient care, and improved professional satisfaction, while the disadvantages include hierarchical and territorial suspicion, professional isolation, and diminished clinical ownership.7 The effects of clinical audit have been small to moderate, but potentially worthwhile.8,9 Similar results have been detected in effectiveness studies on nursing QI/audit.10–,12

The belief that different QI models are effective tools for improving patient care is hard to prove.13,14 The absence of distinctive results could be a consequence of a shifting reality, but it could also reflect the methodological problems frequently encountered in these “clinical reality” studies.15 QI is a complex notion involving philosophical perspectives as well as practical tools for improvement, and results in studies that may evaluate an intervention that is incompletely implemented.13,16 Failure to complete all stages of the quality cycle may have a serious effect on the capability to create change and improvements.17,18

Supporting factors and obstacles to QI/audit

Despite (or maybe because of) the absence of strong evidence for positive effects of QI/audit, there has been considerable interest in studying what enhances or impedes the processes of QI in health care. Many cultural and organisational factors are widely recognised as crucial for success when implementing QI programmes including strategic leadership, supportive organisational culture, effective training, dedicated staff, protected time, and structured programmes. QI has also been recognised as a means for professional development and accountability. Legislation calling for QI in health care is also a supporting factor. There are also several potential barriers to QI/audit including confusion resulting from the many different approaches to quality, deficiency of resources, lack of expertise in methods and project management, problems with interaction and group processes, the absence of an overall plan, and organisational impediments.7,15,19–,21

Studies focusing on the involvement of nurses and therapy professions in clinical audit have reported a large variation in activity. Enthusiasm is mixed with several obstacles—for example, hierarchy and organisational problems, workload pressure and deficiency in protected time, lack of staff commitment, poor knowledge and skills, as well as availability of practical support.11,22–,25 Ownership for quality and action to improve are identified as two key factors for the successful implementation of a quality system.26

This process of “trial and error” for establishing QI has also been observed in Sweden. In 1993 a national regulation was launched, focusing on systematic activities for QI and making all health professionals accountable for appropriate initiatives. A revision of the regulation 3 years later further underlined the system approach and management responsibility.27 Since then QI has gradually been established, employing several different approaches from total quality management (TQM) to more unit based initiatives. A total picture is lacking, however, and QI activities vary a great deal between healthcare organisations.28 With regard to nursing, there was an initiative from the Swedish Society of Nursing in the early 1990s concerning the implementation of the Dynamic Standard Setting System (DySSSy) and, in recent years, more emphasis on clinical guidelines and indicators for nursing care has evolved.29

It is obvious that the implementation and accomplishment of QI programmes is far from straightforward. The authors were presented with an opportunity to study this problem in a follow up study of nurses who had participated in basic training courses in QI. A natural experiment evolved when the nurses were exposed to different contextual conditions for their subsequent involvement in audit (fig 1).

Figure 1

Overview of study design. TI=targeted intervention; GE=generic education.

Study intervention

The intervention, targeted at a group of neonatal nurses (TI group), involved the participation in a national joint project of the Swedish Society of Nursing and the Specialist Group of Paediatric Nurses to establish QI activities and develop national guidelines for neonatal nursing. A framework of topics for the guidelines was agreed upon, followed by a 2 year period of local audit work at 42 neonatal units (at each unit a group aimed to develop, implement and evaluate a standard of care). This work was facilitated through continuing personal contacts (mainly by telephone and fax) between the project leader and the unit based group leader. Networking was established within the guideline project by four meetings of the local group leaders during a 2 year period. Experiences from the local audit work and further literature studies composed a platform for producing the national guidelines. This work is described elsewhere.30

The other nurses, comprising the generic education (GE) group, had no national context for their audit activity. They did not receive national or external facilitation, nor was a national network built up for them, so these nurses had to manage audit with the available facilitation and leadership support in their unit or department. The GE group was considered as a control group.

All the nurses in the study had undertaken basic training courses which gave them a common level of knowledge and understanding about QI/audit. Rather than investigating the effect of training which is supposed to be essential for QI/audit activities,31 we wished to evaluate the support strategy for nurses working with QI in the guideline project by studying the progress of audit, sustainability of audit work, and identifying facilitative and inhibitory factors for audit performance.

METHODS

Setting

From August 1993 to May 1994 the Swedish Society of Nursing arranged eight uniformly designed 4 day basic training courses for nurses in order to manage a method for quality improvement. The method, the Dynamic Standard Setting System (DySSSy),32 has a problem solving, patient centred, team based approach directed towards the development, implementation, and evaluation of potentially better practices (box 1).

Box 1

The Dynamic Standard Setting System

The Dynamic Standard Setting System (DySSSy) is a criterion based approach to clinical audit that was developed by the Royal College of Nursing of the UK in the late 1980s. It evolved from the integration of several concepts of healthcare quality, including the elements of a QI cycle, with descriptive, evaluative and action phases, Donabedian’s dimensions of structure, process and outcome, and the idea of standards as summary statements with more detailed descriptors (criteria) of how to achieve them. Underpinning the DySSSy framework are important philosophical beliefs about creating and maintaining quality systems in health care. These include the need for a patient focused, practitioner (or unit) based approach to QI, supported by concepts of collaboration and multiprofessional teamwork, organisational leadership and support, topic specific problem solving for improvement, and a facilitative approach to implementation in practice.23

Sample

The participants in the eight courses were identified from a register held by the Swedish Society of Nursing. All were registered nurses working in Swedish health care, and all were invited to take part in the study.

The sample originated from two groups. The generic education (GE) group consisted of 156 nurses from more than 50 healthcare institutions representing various clinical areas such as acute care (university, county and local hospitals), psychiatry, primary care, and nursing homes. Twenty of them were not accessible because of retirement or because the postal address was not available. The targeted intervention (TI) group included 84 nurses from 42 neonatal units.

Questionnaire

A questionnaire was developed covering background variables (individual and unit related) and outcome variables on the process of changing clinical practice. It was mainly made up of items with fixed response alternatives, including a few open ended questions (table 1). The items were developed from a review of the relevant literature, then subjected to face validation by two experienced researchers, and finally tested on nurses with a corresponding background to the respondents in the study. The main outcome measure was progression in the QI cycle which itself exists in many versions. We refer to Deming’s four stage model, the “Plan-Do-Study-Act” cycle (PDSA),33 which is similar to approaches used to quantify the success of QI in other studies.17,18 Data were collected 4 years after the basic training courses were completed.

Table 1

Overview of questionnaire used in the study of progress and sustainability in audit

Data analysis

Fixed response alternatives were dichotomised into either positive or negative responses. To assess audit progression and continuation in audit work, the relationship between background and outcome variables was analysed using χ2 tests and logistic regression. Independent variables for logistic regression were selected by identifying significant variables in χ2 tests between the outcome variables and each background variable. Co-variation between selected background variables was also tested; p values of <0.05 were considered significant. Rank order for reported reasons concerning continuing/discontinuing audit was obtained by simple addition.

RESULTS

Response rate and exclusions

The response rate was 79.8% (67/84) in the TI group and 63.9% (87/136) in the GE group; four questionnaires were returned blank.

Because QI is an explicit part of the management responsibility, all nurses in managerial positions at departmental level were excluded from the study (fig 2). The sample therefore consisted of 66 nurses in the TI group and 70 nurses in the GE group; both groups were exclusively made up of staff nurses and nurse managers at the unit level (table 2).

Table 2

Comparison of background variables in the two sample groups at the time of the survey

Figure 2

Flow diagram showing the number of participants in the study.

Differences in audit progression

Nurses in the TI group completed the audit process significantly more often than those in the GE group. Throughout all phases the nurses in the TI group had made greater progress in implementing change in clinical practice (χ2=20.0, df=3, p=0.0002; fig 3).

Figure 3

Comparison of nurses in the targeted intervention group (TI, n=66, 1 missing) and the generic education group (GE, n=70, 7 missing) on level of progress in unit based audit (related to the PDSA cycle). The level of progress is “backwards” accumulated. All four stages: χ2=20.0, df=3, p=0.0002 (χ2 test based on the numbers reaching each individual stage, not on the accumulated numbers in the figure). Dichotomised Plan/Do v Study/Act: χ2=16.8, df=1, p<0.0001.

The stages were defined to represent progress through the QI cycle as follows:

  • Plan: selected a topic and worked on producing a standard.

  • Do: implemented a standard in clinical practice.

  • Study: evaluated clinical practice.

  • Act: accomplished changes in clinical practice based on the evaluation.

The four stage cycle typically took 1–2 years to complete. In the analysis the four stages were dichotomised (Plan/Do v Study/Act) to make the differences in audit progress clearer: 63.1% of the nurses in the TI group and 27% of nurses in the GE group completed the measuring or taking action phases (χ2=16.8, df=1, p<0.0001).

Data were collected on background variables concerning the current conditions for working with QI at the time of the survey (table 2). The two sample groups differed in three of the 10 variables: full time/part time position (χ2=12.2, df=1, p=0.0005), facilitation resources at the departmental level (χ2=14.8, df=1, p=0.0001), and continuing to use the same QI method that was learned during training courses (χ2=7.5, df=1, p=0.006). Logistic regression was used to test the relationship between the nurses’ level of progress in audit (dichotomised Plan/Do v Study/Act) and the independent variable (TI v GE group) and background variables. Only the independent variable was significant, indicating that the targeted intervention accounted for the differences in progress of audit (β coefficient=1.452, p=0.003, odds ratio=4.27, 95% CI 1.65 to 11.05).

Changes in professional roles

As there were some changes in professional roles during the time of the investigation (1994 v 1998), an additional exclusion was made of the nurses who had been appointed to managerial positions at the departmental level during these years (one in the TI group and 16 in the GE group, fig 2). The final sample sizes for analysing the sustainability of QI work was 65 nurses in the TI group and 54 in the GE group. The most apparent difference was the turnover rate; 34 nurses in the GE group (48.5%) had changed workplace compared with 11 (17.1%) in the TI group. There were also some major changes regarding their professional positions (24 (34.3%) in the GE group had changed position compared with six (9.0%) in the TI group). Sixteen nurses in the GE group had left for positions as managers at the departmental level or other administrative functions. In the TI group the primary movement was between positions as nurse and nurse manager at the unit level.

Continuation in audit

The respondents were asked at the time of the survey, 4 years after the basic training courses, if they were still actively involved in audit related activity. Of the 119 nurses left in the sample, 46 (39.0%) had continued with audit. There was no difference between the two groups with 38.5% and 39.6% in the TI and GE groups, respectively, still working in QI projects.

Facilitating factors and constraints on sustainability in clinical audit

The relationships between continuing and discontinuing to work with audit and five selected background variables were analysed using logistic regression. The analysis showed that three factors were significantly related to sustained involvement in audit: (1) remaining on the same unit (OR 11.3), (2) taking courses in nursing science (OR 4.1), and (3) maintenance of the same QI model as in the training courses at the unit level (OR 3.1; table 3).

Table 3

Logistic regression indicating factors significantly related to continuance of work with QI/audit

All the respondents were asked to state their reasons for continuing or discontinuing their involvement in audit. The most frequently reported motives for continuing were related to the opportunities for enhancing knowledge, influencing clinical practice, and developing as a nurse. The main reasons for ending their involvement in audit activity were organisational change and restructuring, a lack of facilitation and knowledge, and leaving the workplace (table 4). The reasons for continuing/discontinuing were similar in the two groups.

Table 4

Reasons reported for continuing or discontinuing QI/audit

DISCUSSION

The TI nurses completed the audit process significantly more often than the GE nurses. There was, however, no difference in their longstanding involvement in audit work. In the following discussion we highlight components of the intervention, the long term effects, and motivating/impeding factors for involvement in QI.

Audit progress linked to the intervention

As in many other studies, the success or progress of clinical audit was highly variable among the nurses in this study.11,22,24,25 A difference between the two groups in terms of progress with the audit cycle was evident, however. The nurses from the national project (TI) completed all phases of audit to a significantly higher degree than those in the GE group; 45% completion of the quality cycle is comparable or higher than that reported in other studies.17,18,20,25,34 Despite shortcomings in the study design, there are reasons to believe that the targeted intervention was an important factor influencing the outcome. Three components of the intervention were particularly important in relation to the results—a common focus for improvement, networking, and facilitation.

Common focus for improvement

The collaborative context of the national project served as a powerful impetus for audit performance at the local level. The nurses in the TI group were from the same speciality and had colleagues throughout the country striving for a common objective—namely, to contribute to the development of national guidelines. The basic training course for the TI nurses involved discussions on how to organise and conduct the national project, including negotiations of a timetable for the work. The ownership and selection of topics were prominent features of the QI work in the project.26 Nurses in the GE group were more heterogeneous and lacked this kind of overall plan.7 Furthermore, there was local support for the nurses in the TI group as both nursing and medical managers were requested to facilitate their unit’s contribution to the national project.7,19,35 The status of a national project probably reinforced the enthusiasm and commitment of the participating nurses. The local QI projects did not only contribute to their own unit but were also linked into a wider context. Motivation is known to be an essential factor for involvement in and completion of audit.7,11,21

Networking

The national project is likely to have facilitated the progress of local audit for the TI nurses as it supported collaboration between individuals and between groups in the project. The local teams were connected to each other by the four meetings held during the project (two each year for the nurse leading the group in each unit) and there were good opportunities for networking on these occasions. One of the meetings was topic focused and an expert in each field of interest (usually a nurse researcher) provided feedback on the work of the team. Networking developed between several of the participants working on similar topics. In contrast, the GE nurses had no opportunity for networking.

Benefits such as improved patient outcomes and cost savings have been reported from collaborative approaches to QI—for example, the “breakthrough series”36 and in neonatal intensive care.37 However, collaboration does not provide an automatic route to success; the diversity of sites involved and the necessity of making realistic improvement projects can make it hard to generalise and learn from each other.38

Facilitation

Our data further support the notion that it is most likely the facilitation provided during the project contributed to the differences between the TI and GE groups. For a 2 year period audit facilitation was available in the project. The project leader (LW), a neonatal nurse manager with some training as a facilitator, served as an external facilitator to all local teams as part of the targeted intervention. The support was tailored to the specific needs of the nurses in the TI group, principally task oriented, and was aimed to support the local teams in producing, implementing, and evaluating a standard of care.20,35,39 Because researchers were involved in the project, it also provided the teams with expert knowledge in most of the topics covered. The facilitation was given in different ways, mainly through personal contacts between the project leader and the unit based group leader, and covered such issues as group dynamics, appraisal of evidence, reviewing team produced standards, measures, data collection, analysing results, and composing reports. In addition to the project meetings, telephone and fax were the most common means of communication. The project leader was also invited to visit nearly half of the units and to participate in group meetings.

Many reports highlight the need for appropriate facilitation for learning and development; insufficient expertise has also been identified as a major constraint on progress.7,22,24,35,39–,41 However, the concept of facilitation is heterogeneous and further exploration and evaluation of the contribution of the facilitator is seen as an important issue for better understanding and enhancement of the improvement process in health care.

Sustainable effects

Thirty nine per cent of all the nurses sustained their involvement in audit activity during the 4 years from the training courses to the time of the survey. The national project had finished 2 years previously. Access to networking and external facilitation was thus similar in the GE and TI groups, which may help to explain why there was no difference between the two study groups.

There is no relevant comparative material to judge expected sustainability in working with QI. A diminishing involvement seems common. Thus, the present finding that four of 10 nurses were still actively working on improvements in patient care 4 years after the training course may be interpreted as a favourable outcome. It is also important to bear in mind that the results of our survey consider the individuals, not the whole QI activity at unit level. From another study performed at the same time as this survey,30 we have information that QI activities were taking place at 69% of the units where TI nurses were employed. Engagement may have spread to other colleagues, hence measuring involvement in QI at the individual level may be misleading. Team based activities, including those involved in applying the DySSSy method, could also have moved on to more organisational models of QI. The TQM approach became relatively widespread in Sweden during the end of the 1990s.

Motivation versus obstacles

Three factors were significantly related to sustained involvement in QI/audit (table 3). These factors range from apparently logical relationships to more complex thought provoking ones. Moving to a new job can obviously delay or inhibit active involvement in audit for the individual, implying that a high turnover rate can seriously diminish QI at the unit level. Another more interesting factor involves the relationship between intermediate courses in nursing science and audit activity. It could be that individuals want to learn more when starting to work with QI, or that they desire to change clinical practice when acquiring more knowledge. In either case, this relationship merits further investigation in terms of achieving QI in practice. Maintenance of the QI method at the unit level was the third factor related to sustained involvement in QI/audit. We do not believe that the use of the DySSSy method in itself explains the outcome. Instead, this factor indicates the importance of maintaining a consistent approach to QI in terms of sustaining audit activity at the individual level. Confusion resulting from the many different approaches to quality can be assumed to be counterproductive.21 There may also be problems in keeping up people’s interest when turning to other more organisational models of QI. For future developments we believe that both the individual and organisational ability to integrate elements and build upon different approaches to QI need to be improved.14

Besides the factors described above, generated by statistical analyses, the nurses reported their own reasons for continuing or discontinuing their involvement in QI/audit (table 4). The predominant motives for continuing are, to some extent, homogenous. Audit is a way to influence practice and it gives nurses individual benefits by helping them to enhance their knowledge and develop as a professional. These findings are consistent with other studies7,20 and suggest that QI has rewarding elements for the practitioners over and above improved patient outcomes. To “influence practice” most probably implies a desire to improve the quality of care, but it can also be an issue of power. Studies in the UK have shown positive attitudes among nurses to audit that, to some extent, differ from the attitudes among physicians.22,42 QI may provide nurses with an increased freedom of movement and an opportunity to influence changes in care.13 In contrast, the medical profession has often held that diminished clinical ownership is an effect of QI and thereby a constraint to QI/audit.7 This is an area of great importance as improvements in care are dependent on multiprofessional collaboration. Experience of multiprofessional QI work was also close to being a significant variable for sustained involvement in QI in our study (table 3).

The reasons for discontinuing involvement in QI/audit present a more mixed picture. It is not surprising that organisational obstacles top the list. Changes in organisational structure as well as a lack of facilitation and knowledge on how to handle change processes are commonly reported obstacles to QI/audit.7,22,24 It can be concluded that the importance of a supporting organisational culture is evident for several of the reasons reported. Empowering elements for the individual nurse should, however, not be underestimated if a longlasting involvement in QI is desirable.

Strengths and limitations of the study

To evaluate audit and other QI projects is a complex undertaking. Our survey could have been more informative if supplemented with items covering issues such as the evidence base for change, performance of re-audit, and attitudes to QI/audit.43,44 The main measure used—completion of the quality cycle—gives a useful indication of the quality of audit activity.17,18 However, we were not able to assess whether there was also an improvement in patient care.

The natural experimental design has potential for bias at the same time as it benefits from being rooted in reality. For this study the two groups were sufficiently comparable. Level of education and number of years of professional experience were equal in the groups. QI had not been an issue in basic professional training for any of the nurses. All participated in uniformly designed training courses for similar reasons—they were sent from their unit to learn something new and then returned to start a QI project. The intervention group was not self-selected but consisted of teams from all neonatal units in the country. The training and subsequent audit were performed during corresponding periods for the two groups and therefore historical events should have influenced both groups equally. The neonatal sector of health care was not exposed to any particular quality initiative. The guideline project had been started as an individual initiative, not as part of a national action.

Key messages

  • A common focus for improvement, facilitation, and opportunities for networking seems to enhance the ability to carry out the process of QI but not to sustain the QI work over a longer period.

  • Four out of 10 nurses were still actively working on improvements in patient care 4 years after training courses.

  • QI may provide nurses with an increased freedom of movement and an opportunity to influence changes in care.

  • Empowering elements for the individual nurse should not be underestimated if a longlasting involvement in QI is desirable.

The 64% response rate of nurses in the GE group could be a confounding factor. We assume, however, that the non-responding nurses were those who had less interest and experience of audit. It is possible that the differences between the two groups may have been even greater with a higher response rate. We therefore believe it is justified to draw the following conclusions from the results:

  • Participation in a national QI project—including a common focus for improvement, facilitation, and opportunities for networking—enhanced the ability to carry out audit but not to sustain a QI team over a longer period.

  • All three of these factors are likely to be of importance for successful implementation of QI/audit.

  • A common focus for improvement and networking were evident components of the national project.

  • The tailored approach and the intensity of the interaction between the facilitator and local teams suggest that the facilitation provided to the intervention group was also a determining factor for the outcome of this study.

REFERENCES

View Abstract

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.