Purpose of study

A baseline assessment of the progress with clinical governance in the West Midlands and South West regions of the English NHS was undertaken in March–June 1999 to assess progress towards the end of the first year of the policy initiative.³ This paper reports on the use of organisational strategies to improve clinical practice. The questions the study sought to address are:

• What strategies were they using to change clinical practice?

• How effective did they rate these strategies to be?

• Was intended future use associated with past use and perceived effectiveness?

• What were the expected outcomes of clinical governance in the trust?

• What barriers were experienced in implementing clinical governance?

Answers to these questions are intended to inform leaders of what is perceived to work, why, and how problems might be overcome.

Methods

A questionnaire was piloted on four trusts in Oxfordshire (that is, outside the target regions). The questionnaire was administered by post to the clinical governance lead during March–June 1999. Non-responders were followed up. The questionnaire covered policy and process developments reported by Latham et al.³ The final section reported below consisted of closed questions used to elicit:

1. use: “Have you used this approach to changing clinical practice in your trust?” answered “yes” or “no”;

2. effectiveness: “How effective has this been in changing practice?” answered “not at all effective” to “very effective” on a five point scale; and

3. future intended use of 13 organisational strategies: “How likely are you to use this approach in the future?” answered “very unlikely” to “very likely” (box 1).

Answers to the questions “Please describe one example of particular success in bringing about changes in clinical practice in your trust” and “What do you see as the main barriers to the development of clinical governance in your trust?” were grouped thematically. Finally, respondents were asked “Please indicate whether you think the following outcomes will be achieved through the development of clinical governance by scoring each one from 1 (very unlikely) to 5 (very likely)”. The outcomes to be rated were: reduced patient complaints, variation in clinical practice, use of ineffective investigations and treatments. They were also asked to rate if they expected increased efficiency and better use of resources, increased patient satisfaction, documented changes in clinical practice, closer working between managers and clinicians, more open organisational culture receptive to change, and specific improvements in patient outcomes.

The numerical results were tabulated. Answers to the questions about organisational strategies were compared to determine the relationships between use, effectiveness, and future intended use and compared visually using scattergrams. These plotted the percentage of respondents who indicated that they had used the strategy against the percentage of respondents who planned to use the strategy in the future (likely or very likely use). A second scatterplot was constructed with current effectiveness (effective or very effective) against planned future use. Thematic results of the open ended questions and the ratings of the likelihood of outcomes from clinical governance are presented in tabular form.

Results

Responses to this section of the questionnaire were received from 70 of 86 clinical governance leads (81%). As not all respondents answered all questions, the response rates are sometimes for a smaller subsample. The leads in all but two trusts were senior clinicians, predominantly from the medical or nursing professions.

USE OF ORGANISATIONAL STRATEGIES FOR CHANGING CLINICAL PRACTICE AND PERCEIVED EFFECTIVENESS

The results show that all respondents claimed to have used at least one of the 13 organisational strategies (table 1). Most clinical governance leads reported use of most strategies, except for financial incentives which were only used by 20 of 68 trusts (29%). Inspection of the results by individual respondents across all 13 strategies also showed that five respondents endorsed none of the 13 strategies as at all effective.

The strategy most widely rated as effective was the creation of special clinical groups or teams to focus on a particular problem or issue, followed by the use of pilot projects which may, in some trusts, have referred to the same initiative. The strategies rated as being of poor or equivocal effectiveness by over half the trusts that used them were the educational approaches. The strategy most often rated as ineffective by those that used it was training clinicians in information management, which was rated ineffective by four out of five trusts. However, these educational approaches were also the most widespread in use. Of the facilitative approaches, the use of external sources of advice was felt to be largely ineffective by 31 of 55 trusts (56%) that claimed to have used them. The production and dissemination of protocols was reported to be ineffective by about half of those who used them. Performance management methods were used by about two thirds of trusts, yet the use of feedback of performance data was felt to be largely ineffective by 34 of 46 user trusts (74%), and departmental review was considered largely ineffective by 24 of 43 user trusts (56%). Financial incentives were little used but were found to be effective by 12 of 20 trusts (60%). The use of specific organisational change management methods were reported by two thirds of trusts. They were found to be ineffective in 22 of 49 (45%) and the use of combined methods was rated ineffective by 23 of 41 (56%).

Thematically analysed answers to the open ended question that described one example of success in changing clinical practice are presented in table 2. They show reliance on protocols and guidelines, feedback of performance data, and a number of changes in service delivery. Recent successes seemed to favour “hands on” facilitative approaches.

FUTURE USE OF STRATEGIES

The responses in this section are based on the data provided by fewer respondents than for the previous section (55–63 respondents, depending on the item). Inspection of the answers revealed that those who did not respond were more likely to have indicated they had not used the strategy to date, although clearly some felt able to make predictions even in the absence of current experience of the strategy within their trust. All who responded indicated they planned to use at least one of the 13 strategies.

Approximately 85% of the overall sample expected to use both of the educational approaches in future, and 55 of 63 trusts (87%) expected to use educational programmes. At least two thirds of the trusts planned to use the six facilitative approaches to best practice through clinical leadership. Of these, over 90% expected to use special clinical groups or teams, pilot projects and peer review, while 93% (58/62) planned to produce and disseminate protocols and guidelines. It is possible that these were the same initiatives in most of these trusts—for example, an initiative could have been to undertake a pilot project with a clinical group to adopt guidelines and undertake peer review of practice against the guidelines. External facilitation and opinion leaders were favoured by about three quarters of trusts.

Performance management strategies were expected to be used by about three quarters of responding trusts (75% (44/59) performance data and 72% (42/58) departmental performance review). Again, these strategies may overlap considerably within each trust—for example, a trust could have been using externally benchmarked data as a means of departmental performance management. Only 18 trusts were likely to use financial incentives in the future.

Of the change management methods, combined methods were expected to be used by 47 of 55 trusts (85%), and specific change management methods aimed at identifying and overcoming barriers were expected to be used by 31 (56%).

RELATIONSHIPS BETWEEN EXPERIENCE OF USE, PLANNED FUTURE USE, AND PERCEIVED EFFECTIVENESS

Scattergrams were generated for responses to questions on the use, effectiveness, and planned use of the 13 strategies. The results, based on the percentage of affirmative responses obtained from the total sample of 70 trusts, show that planned use of interventions in the future is strongly related to past experience of use (fig 1). However, fig 2 shows that planned future use is only weakly associated with judgements of the effectiveness of the intervention method in changing clinical practice. Indeed, there seems to be unjustified reliance on methods such as peer review, the use of performance data in feedback, and in training clinicians in information management, given the low ratings of perceived effectiveness.

BARRIERS TO THE DEVELOPMENT OF CLINICAL GOVERNANCE

The results in table 3 are the answers to an open ended question about the barriers to introducing clinical governance and represent the responses of 68 participants. The overwhelming result was the perceived lack of resources of all types, but mainly money, time for clinical staff to participate, and manpower. The paucity of IT, information, and library systems was a related resource issue but was less frequently described. The second most frequent responses described the need to establish both a “no blame” culture and to ensure that clinicians are accountable for the quality of care. These latter points would seem to be those most appropriately addressed by organisational change interventions as well as by interventions designed to improve effective clinical practice.

OUTCOMES OF CLINICAL GOVERNANCE

Nine objectives were suggested for clinical governance, five of which were directly consequent upon change in clinicians' behaviour (table 4).

Clearly, more than half the trusts expected to achieve an impact on clinicians' behaviour. The most widely expected outcome was documented changes in clinical practice by 59 of 69 trusts (86%). Fifty four of 70 trusts (77%) expected reduced clinical variation while 23 (33%) did not expect reduced ineffective investigations and treatments and only 37 (53%) thought that a likely outcome was better use of resources. The closer relationships between managers and clinicians expected by 59% (41/69) and the more open culture receptive to change anticipated by 68% (47/69) suggested a positive attitude to the attitudinal impact of clinical governance. The expected impact on patients was less positive; 78% (54/69) expected there to be improved patient outcomes, 36% (25/69) expected improved patient satisfaction, and only 10% (7/70) expected a reduction in patient complaints.

Discussion

This survey is a snapshot of the use of organisational strategies to modify clinical practice by nearly one quarter of the NHS since it covers two of the eight regions in England. There is evidence that all trusts have used one or more organisational strategies, with only the use of financial incentives being confined to a minority. The ratings of effectiveness show that facilitative methods such as creating special clinical teams to address particular issues and pilot projects stand out as most often being rated as effective. In contrast, widely used educative methods such as training clinicians in information management were said to be effective by only one in five respondents. The results from descriptions of one successful example per trust were somewhat different, with most describing the use of protocols and guidelines and the use of performance data as feedback. Intended use seems to be associated with past use rather than ratings of effectiveness. The barriers to implementing clinical governance centre around resources rather than cultural issues. The expected impact of clinical governance included care being better documented, but only half expect that it will improve the use of resources, a third do not expect it will reduce ineffective practice, and 20% do not expect clinical outcomes to improve. Nearly two thirds expect there to be better relationships between managers and clinicians and a culture more receptive to change, but only 10% expect patient complaints to be reduced.

The limitations of the study must be recognised. Response rates of around 81%, while high for a postal survey, are not complete for all sections of the questionnaire. There was a lower response rate in the South West region, probably because the project was led by the West Midlands region which also linked the research to a regional visiting team process to each of its trusts. The methodology relied upon clinical governance leads to complete or to ensure completion of the questionnaire. It is unknown therefore how authoritative the respondents were. Given that clinical governance leads are generally board level staff or report to board executives, it is reasonable to expect that they would be aware of organisational strategies to promote changes in clinical practice. Self-report methods introduce subjective response biases, not least the social desirability bias towards “faking good”. However, it is also true that comments to open ended questions revealed in many cases a degree of self-criticism, but the extent of honest reporting may be unevenly distributed. The research was not able to verify reported use of strategies and their actual effectiveness on clinicians' behaviour nor on clinical outcomes. Future research would be able to answer more fully the question of what works if it is prospective, has objective measures of the use of strategies, and has measures of clinical behaviour and clinical outcomes.

The results of the study throw some light on how far the interventions and their use reflect the systematic evidence based approach described by Ashford et al.⁷ Questions were not asked in this study to ascertain whether trusts based their plans for future change strategies on either a diagnostic review of what interventions had been effective locally or by research evidence of effectiveness. Given the clearer relationship between planned use and past use, rather than with perceived effectiveness, it does appear that the choice of method is influenced more by what the trust is already comfortable with than by evidence of what works. A case in point is the discrepancy between past experience of effectiveness and predicted use in relation to the use of performance management strategies. While about two thirds of trusts claimed to use some form of performance management and expect to do so in the future, less than half believed these methods to be effective. In contrast, one of the least used methods—and for which research evidence is sparse—is the use of financial incentives.¹¹ Only 20 trusts claim to use financial incentives, but as many as 12 claim them to be effective. More detailed research is required to determine what type of incentives are used, whether they are individual or team based, imposed or negotiated, and whether they lead to unintended behaviours arising from perverse incentives. Particularly worrying is the finding that five trusts did not find any of the approaches listed as being at all effective.

A great challenge for the NHS is to learn not just how to change clinical practice, but how, in doing so, the organisation can foster a culture that mobilises its many constituents to contribute their implicit knowledge and skill so that the organisation learns to learn. The NHS Executive guidance¹ called for “a fundamental shift in the culture of many NHS organisations” and “an open and participative culture in which education, research and the sharing of good practice are valued and expected … a commitment to quality that is shared by staff and managing culture in a systematic and demonstrable way so that quality infuses all aspects of the organisations' work”. The results of this survey show that a positive impact on management-clinician relationships and culture was expected by at least half the trusts. This will not be easily attained, given that no specific change management techniques were planned by about half the trusts. On the other hand, it could be that leaders intended to take a more subtle line and to rely upon diffusion from the likely positive impact of the preferred facilitative strategies. However, experience in organisation development suggests that, to maximise the impact beyond the “early adopters”, other more systematic approaches are needed.⁸¹²¹³

There was evidence of ambivalence by trust leaders about the expected outcomes of clinical governance, particularly those concerned with patients' expectations and complaints. It may be that the barriers were seen to be daunting for some, and the expected benefits too mixed. While most respondents anticipated improved patient outcomes, very few believed clinical governance would improve satisfaction and reduce complaints. This suggests that trusts were aware that other factors—such as the expectations of patients, the style of consultation, and the attitude of professionals—may have to change which would require additional effort and resources.

Conflict between expected outcomes and the anticipated effort to achieve a desired impact may create dilemmas for trust leaders, and future research should examine the effort:output ratio for managerial decisions. Expectancy value theory¹⁴ and subjective utility theory¹⁵ provide predictive models based on assumptions of subjective judgements of the value of the intended outcome (utility) and the probability of actions achieving these outcomes (expectancy). The relative value of these judgements may be used to explain how leaders choose how and when to adopt innovations. These models suggest that trust leaders' subjective judgements of the effort involved and of the likelihood of benefits and adverse reactions may influence the adoption process. Some outcomes of clinical governance may not be entirely attractive, particularly if the new practices incur higher costs, and this perception may help to explain why some trusts appear to be continuing to adopt methods of low perceived efficacy. The continued use of ineffective strategies may result from the judgement of trust leaders that the effort involved in using low efficacy methods is low, but satisfies a need to at least appear to be taking some action. These questions cannot be answered in the current study.

In conclusion, trusts seem to have been backing approaches that rely upon clinicians acquiring best practice through multiple optional means, and expecting organisation wide change to occur through some unspecified diffusion process. It is doubtful whether this is sufficient to achieve a comprehensive improvement in quality across all systems and in all trusts as is clearly intended by this policy. A more systematic and explicit approach to organisational learning is required. Clinical governance leads could show leadership by example in evaluating explicitly the efficacy of the strategies used to influence clinical practice to help instill an evaluative learning culture.