Clinical audit is seen as one approach to improving the quality of patient care.^{1, 2} Its development in the UK was linked to clinicians' desire to improve medical care.² It was thought that, by drawing attention to deficiencies in the delivery of care, this would curb inefficient and ineffective practice.² Clinical audit was introduced throughout the UK NHS in 1990, but the introduction of audit in such a way was untested. Its introduction was based more on faith on what it might achieve³ (box 1). There was little evidence to suggest that there would be definite benefits that would justify the scale of the investment.^8–11

No agreement existed as to which audit methodologies were the most suitable approach and, not surprisingly, there was significant confusion among healthcare professionals about how to implement audit and integrate it effectively into clinical practice.^{9, 12} There has been a demand for audit programmes to be better evaluated and to be more accountable.^{9, 10, 13–15} Concerns about the effectiveness of clinical quality improvement activities are not confined to the UK and have been expressed elsewhere in the world—most notably in the United States.¹⁶

Despite this lack of objective evidence supporting its value, the need for high quality audit has been given fresh impetus through the advent of clinical governance.⁶ The term clinical governance has been recently introduced into the UK NHS to indicate integration of clinical quality improvement with organisational and service performance at all levels. Clinical quality will now be integrated with financial control and service performance at all levels if organisations providing health care are to be managed well.¹⁷ Hospital chief executives will be personally responsible for the clinical performance of their services.¹⁸ The centrality of clinical audit in this process is therefore reaffirmed. Clinical audit is now rooted in professional practice within the UK NHS and will be an important aspect of clinical governance. Clinical audit will be central to this process. Indeed, recommendations to facilitate the dissimulation of the concepts of clinical governance into everyday practice seem to reiterate those already advocated for audit.^{19, 20} Moreover, it could be argued that audit is one of the few mechanisms with common elements equally applicable to clinical decision making and to organisational efficiency. This suggests that, for clinical governance to achieve the aims of continuous quality improvement, more attention needs to be paid to clinicians' views about audit and to those factors which have been shown both to allow audit to develop and those that have been shown to impede its progress.

This article presents a review of the literature evaluating clinical audit. The number of papers reflects the now considerable range of clinical audit activity being undertaken among healthcare professionals. The variety and scope of studies illustrates the gradual evolution of audit from being a specialised activity exclusive only to doctors to one which is much more widespread, encompassing every discipline and every grade. Yet, findings also suggest that the depth of involvement across health care is still uneven—significant activity is still largely confined to enthusiasts.

Aim

This article seeks to use a comprehensive review of the literature to (a) identify the main advantages and disadvantages of audit and (b) assess the barriers and facilitating factors which either impede or promote successful audit. Much of the literature is from UK studies, although studies from other countries have been included when these provide additional insights. The aim was to determine what recommendations could be made to make clinical audit more effective.

Methods

A search was conducted of Medline and CINAHL databases for the years 1992-7 using the keywords of “audit”, “audit of audits”, and “evaluation of audits”. Papers which addressed empirical evidence from studies of clinicians' views, and also theoretical discussions, were retrieved and included in this study. The indexes of the BMJ, the Lancet, British Journal of General Practice, Medical Audit News, Quality in Health Care, Audit Trends, and Medical Education were also handsearched for key articles. Each article was carefully read by one of the authors (GJ) and its key findings were identified. The findings were organised into coherent themes using a traditional narrative review approach. This process used a largely qualitative approach and its aim was to identify common elements in the literature which, when integrated, would lead to a greater understanding of the salient issues surrounding clinicians' experiences of undertaking audit.²¹ Key points from each article were collected and classified into broad categories. These categories were then discussed with the remaining authors and refined and synthesised into major themes. No attempt was made to evaluate methodologies or data from the studies retrieved due to their largely disparate and discursive nature. Classification of the main findings is shown in the appendix.

Although this is a comprehensive review the findings are presented in summary form illustrating all the key points which were identified.

Results

A total of 93 articles were identified. These included discussion papers, government publications, and some papers relating to the same study (appendix). A full bibliography is available from the authors. Studies ranged from single case studies of individual audit projects through retrospective review of departmental audit programmes to studies of interface projects between primary and secondary care. They incorporated the views from a wide variety of clinicians from medical consultants to professionals allied to medicine and from those involved in unidisciplinary and multidisciplinary ventures. The literature review identified 4 main themes:

• Importance of clinicians' perceptions of the benefits of audit

• Importance of clinicians' perceptions of the disadvantages of audit

• Barriers which block its success

• Facilitating factors which promote its success.

HEALTH PROFESSIONALS' ATTITUDES: BENEFITS OF AUDIT

The perceived benefits of audit can be conveniently discussed under the headings of professional benefits, improvements in patient care, and the quality of service provided.

Professional benefits

Several studies have reported that clinicians have felt they had benefited from audit through improvements in communication between professional groups and increased professional satisfaction and knowledge.^22–26 A participant observational study of a series of audits of the management of six conditions was conducted in one group practice to understand the factors facilitating change.²⁷ The study showed the importance of acknowledging the attitudes of those whose behaviour was being audited and modifying the audit process to accommodate them. Changes in prescribing behaviour were attributed to the fact that doctors were able to control the audit process using their own values and attitudes and that being able to compare one's own practice with that of immediate colleagues and outside authorities provided a powerful impetus to changing behaviour. In addition, audit was seen to promote communication between partners and as a stimulus to learn from colleagues' behaviour.

These additional benefits from audit can increase staff enthusiasm. A pilot study of audit in four professions allied to medicine showed that although there were some constraints to its development there was no evidence of a negative or defeatist attitude to audit.²⁸

Patient care and service delivery

Benefits to patient care and service delivery have been commonly identified in audit studies.^{4, 22, 29–33} A postal survey to explore the attitudes of general practitioners to audit found that 68% (n=144) of respondents had had experience of audit and 34% (n=72) claimed to have benefited from it.³⁰ Only 28% reported changes as a result of medical audit, nearly half of which were doctor centred and a quarter patient centred. The patient centred benefits were improvements in patient care, improved patient satisfaction, and better patient feedback. Doctor centred benefits included increased knowledge, satisfaction, performance, and teamwork. Overall, however, attitudes to audit were largely positive and were related to audit experience; the more audit experience the more positive the attitude.

HEALTH PROFESSIONALS' ATTITUDES: DISADVANTAGES OF AUDIT

Support for audit has not been universal and even those enthusiastic about audit have recognised certain downsides. Sellu has suggested that a key reason why audit has foundered is that doctors have not been convinced that it improves quality.³⁴ Perceived disadvantages of audit are diminished clinical ownership,³⁵ fear of litigation,³⁶ hierarchical and territorial suspicions,³⁷ and professional isolation.³⁸

Increased workload

Many studies report that clinicians feel that audit detracts from clinical work at the expense of patient care,^{39, 40} and that collaboration onlarge projects leads to a reduction and de-skilling of practice based activity.⁴¹ Consequently, some feel that audit is a waste of time and effort, and that resources would be better spent on patient care.^{39, 42} Others think it is irrelevant and not part of their job.⁴³ Comments from individual clinicians have described audit as an uninspiring but necessary chore,⁴⁴ and “worthy, high minded, useless”,⁴⁵ or a “fatal flower and with every new blossom it becomes more and more remote from real practice”.⁴⁶

Restriction of clinical freedom

The nature of audit, with its primary aim of improving the quality of care, has important implications for professional conduct. Paradoxically, conflicting perceptions of audit are often evidenced in single studies, highlighting the confusing and often complex issues involved in the way clinicians embrace audit. Although one group practice found many benefits associated with audit, members also perceived it as an external threat and a barrier to providing individualised and the best care possible for patients.²⁷ In this study, doctors also expressed a reluctance to criticise each other's behaviour for fear of causing offence, and therefore there was an unwillingness to use clinical standards against which clinical performance could be judged. For these reasons they refused to adopt a practice prescribing policy. Similarly, a study of medical audit in four hospitals found that while 68% of junior staff surveyed had altered their practice because of audit, consultants were more sceptical and found it difficult to tell whether improvements had stemmed from audit or any other source.³⁸ Some consultants were quite definite that it had taught them nothing new.

Professional threat

It is apparent that while some clinicians are merely unconvinced, others are decidedly hostile to audit. Two large studies of doctors' perceptions of audit have described its negative impact.^{47, 48} Negative attitudes are associated with suspicion about its motives, fears of intimidation, and ridicule; beliefs that it caused discord among professionals; and a feeling that it was being used as a government ploy to discipline doctors and stifle individuality.⁴⁷ Junior doctors in particular saw it in a less positive light than their seniors and described it as threatening, blame apportioning, and a means of professional witch hunting.⁴⁸ Junior doctors have also reported feeling unfairly criticised and alienated while they felt that consultants were not audited rigorously enough.⁴⁹ Many have felt discouraged by the lack of support, direction, and feedback from seniors.^{50, 51} These feelings are exacerbated by short term contracts and an inability to see projects to fruition.^{48, 52} Ironically, consultants have also identified a lack of motivation among junior colleagues as important in the failure to achieve meaningful surgical audit.⁵³

These negative attitudes are not confined to doctors. They are also perceived by other clinicians as well as dedicated support staff.⁵⁴ A study of constraints on the progress of audit experienced by physiotherapists, occupational therapists, speech and language therapists, and clinical psychologists⁵⁵ found that junior therapists in particular felt that audit projects about record keeping were a means of checking up on them. In addition, some feared that the results of audit would lead to a reorganisation of the service and a reduction of their autonomy and even threatened job loss. Important differences in attitude to audit have also been found among different staff groups.⁵⁶

BARRIERS TO SUCCESSFUL AUDIT

Achieving successful audit is not without its difficulties. Given the disparate and divergent views held about audit it is not surprising that there are many perceived barriers to implementing it. In a study of medical audit activity in West Scotland, Kinn and Smith reported a rule of diminishing returns where just half of those involved in audit had completed a project and only half again had repeated a project.⁴³ Aprogramme of evaluative projects to review the progress of audit was commissioned by the Department of Health in 1993. Its purpose was to assess the development of audit and its impact on the quality of care, involving a series of interlinked projects each directed at different areas of the medical audit programme in the hospital and community health services in England.^{22, 35, 57–60} These have shown that the lack of sound methodology used in audit projects resulted in large variations in the approaches taken to audit.⁶¹ Some critics have argued that, to date, audit has led to “spectacularly” few obvious benefits to patients.⁹

The main barriers to clinical audit can be largely classified under five main headings, following the findings proposed by Robinson⁶²: a lack of resources, lack of expertise or advice in project design and analysis, relationships between groups and group members, lack of an overall plan for audit, and organisational impediments.

Lack of resources

Numerous studies have described clinicians' common problems about lack of time and dedicated staff^{40, 53, 63} and inadequate financial and practical resources.⁶⁴ A national study by the National Audit Office for Scotland⁴ found that the major problem which clinicians described was the lack of time to do audit and the resulting conflict between the immediate demands of treating patients and the longer term benefits of audit. Audit facilitators were found to be successful but scarce, so health professionals' time was used inappropriately for tasks which did not use their professional skills. Problems also arose when there was a lack of good quality information systems and information specialists to help clinicians. In addition, there were perceived problems with the financial management of audit funds nationally as well as uncertainty over funding arrangements at a local level. The availability of large amounts of audit funds has also meant that money has sometimes been misappropriated, buying overly complex equipment which staff are not trained to manage.⁵⁷

Lack of expertise in project design and analysis

Many studies have reported the obstacles imposed by a lack of expertise in audit methods. These relate to poor design,^{22, 48} problems with standard setting,⁶⁵ inappropriate and haphazard data collection,^{26, 35} a shortage of good audit tools,^{28, 49, 66} lack of education and training in audit methods,^{22, 35, 52} and a lack of access to skilled and proactive support staff.^{35, 48, 67}

Robinson identified five aspects of undertaking audit projects which required expertise⁶²: writing proposals, specifying and keeping to objectives, designing instruments, analysing data, and writing reports. Sources of guidance specified by those interviewed included colleagues with audit experience, quality assurance staff, dedicated research or audit staff, published work, and audit networks.

However, these difficulties may be compounded by “experts” and support staff who are inadequately trained in audit methods or are too few in number to provide the essential support needed.^{2, 58, 67} In addition, their contribution and status are often impeded by short term contracts which fail to be renewed when funding has run out.^{35, 54, 58} One survey of academic departments found that respondents expressed concerns about teaching audit due to a lack of expertise and knowledge in the subject and a worry about how to make it interesting.⁶⁸ A national survey of 382 audit support staff who had registered with an information service found that the majority were women and employed at ancillary and clerical levels.³⁸ Seventy one per cent had received training in the basics of audit but 29% had had no training. The mean length of training was three days. Nearly half of all respondents felt that their training was inadequate or barely adequate. The study also found that audit support staff often felt devalued by medical staff and forced into a delegated role. They therefore found it difficult to take the lead and influence the design of projects and felt they were perceived as glorified secretaries and report writers. On the other hand it has been found that successful audit programmes were those whose leaders had thought carefully about the range of skills the service needed, recognised that the duties of audit staff were not merely clerical, gave training when required, and treated them as valued members of the team.⁶¹

Lack of an overall plan for audit

The lack of an overall plan for many audit programmes is reflected in the wide variations in the way audit projects are undertaken and a lack of common vision about their goals and purposes.^{35, 58, 59, 61} As a result, their success is often dependent on one enthusiastic leader holding everything together.^{38, 57} However, it also means that well resourced projects can fail to make an impact because of poor links with management,^{35, 55} or that projects fail to be completed because support in terms of time and funding have been badly underestimated by participants.^{22, 35, 55} This means that although some resources are available to support projects, others equally necessary for their completion are missing.⁵⁵

Relationship problems

Dysfunctional group membership or ineffective group dynamics can also impede the success of audit.²³ Poor relations between and within groups resulting from lack of commitment, a failure to include those being audited, changes in leadership, conflict between staff, concerns about confidentiality, lack of ownership, and a reluctance to change practice have all been found to influence the success of audit.^{35, 59, 62} Studies of interface audit where groups comprise members from both primary and secondary care have also shown that the audit process can be affected by fluctuating membership of the group, lack of clearly defined group tasks, and different professional backgrounds causing mistrust, language barriers, and a lack of knowledge about each others' roles.²³ In addition, there are logistical problems of finding appropriate and mutually convenient places for meetings⁶⁹ and in holding meetings, even when membership is unidisciplinary, which are not frequently interrupted by telephones, bleeps, and members coming and going.³⁸

Organisational impediments

The absence of a supportive working relationship between clinicians and managers may also impose organisational barriers to audit and the implementation of findings.^{67, 70} Studies have shown that there may be disparity between the views of clinicians and management,^{53, 59, 71} and a lack of clarity among clinicians about lines of authority and accountability.² As a result, confusion and inertia often exist among clinicians about who should be responsible for making changes—feelings which increase when changes involve negotiation within the wider hospital.^{38, 59}

A survey of managers and clinicians before the introduction of medical audit reported that although the two groups concurred about the potential advantages of audit, many had different opinions about its disadvantages.³⁹ Seventy one per cent of clinicians thought that it would interfere with their clinical workload and 41% that it would consume resources better spent on patient care. Only one out of eight managers shared clinicians' views, and they were less likely than clinicians to believe that audit would allow them to influence medical practice. Similarly, a study of managers, clinicians, and audit leaders found that while purchasers and providers shared common views on the purpose of clinical audit, there were important differences in their views on the level and appropriateness of involvement of healthcare purchasers.⁷² There were also differences of opinion relating to the sharing of information in the outcomes of clinical audit and changes in behaviour, and in the way audit should be structured and integrated into present NHS processes.

A refusal by managers to make changes on the grounds of cost, and unclear directives from purchasing authorities, also serves to impede the potential of audit to change practice.³⁴ In addition, large organisational changes such as hospitals merging or provider units becoming trusts have been shown to delay and disrupt audit activities.²²

PROMOTING SUCCESSFUL AUDIT

Quantifying success

Attempts to quantify the degree to which audit projects are successful based on the extent to which they complete the audit cycle have been facilitated by the development of classification systems.^73–77 Several studies have surveyed audit activity using these systems to determine the degree of success of audit activity and the factors associated with it.

One large study of 169 general practitioner practices from which information was available reported that 26% had completed one full audit and 24% had not started audit.⁶⁴ Using the Oxford classification system, mean scores were significantly higher with the presence of a practice manager, computerisation, organised notes, being a training practice, and being a partnership. In an earlier study of 80 general practices, 58% were classified as doing audit and of audits being undertaken, 54% included planning care or setting standards of care.⁷³ However, a review of audits in an ophthalmology department showed that only one of 18 audits reviewed fulfilled the criteria required to qualify for a full audit.²⁶

These studies show that audits themselves can and should be monitored with the intention of improving the effectiveness of an audit programme. Paradoxically, they may also suggest that it is likely that those organisations most in need of evaluation and with most potential to improve patient care will be the least likely to establish successful audit programmes.⁷⁸

Factors which promote success

Despite many obstacles, evaluative studies of medical and clinical audit programmes have helped to define some of the factors which facilitate successful audit. These include the need for practical mechanisms to make data collection easier, including modern medical records systems^{24, 64}; information technology and improved links between routine data collection and audit²⁶ as well as dedicated staff; and protected time to release the burden on clinicians' clinical workload.^{35, 40} From a review of initiatives, Walshe and his colleagues at Caspe identified key factors which promoted the success of audit.⁵⁷ These included a supportive organisational environment, sound leadership and direction of audit programmes, strategy and planning in audit programmes, resources and support for audit programmes, monitoring and reporting of audit activity, commitment and participation, and high levels of audit activity which by its nature and impact is seen by its participants to be involving and relevant. The extent to which any or all of these factors are present appears not only to affect whether audit actually takes place,^{63, 64, 79} but also determines the degree to which it is successful.^{59, 80–82}

Studies of audit in general practice highlight the importance of information technology and support from colleagues in successful audit. A survey of 54 general practices in West Glamorgan assessed the development of records systems and practice organisation and related this to audit activity.⁸³ It found that practices with three or more partners and modern medical records systems were more likely to be involved in audit. Audit activity had occurred in 87% of practices with long term medication summaries and in 87% which kept clinical summaries and 85% of practices with age-sex registers. All training practices had undertaken audit compared with only 63% of non-training practices. Similarly, a Dutch study of 120 general practitioners found that, in addition to financial support, the main requirements for implementing quality assurance programmes and medical audit were regular meetings with colleagues, information on the aims and methods of quality assurance, support in setting up audit and in data collection, and peer review.⁸⁴

Discussion

THE FUTURE OF AUDIT

This article has identified some of the benefits and some of the difficulties of the implementation of clinical audit and the practical and attitudinal barriers which prevent its progress. At the same time it has drawn attention to some of the main facilitating factors which can make audit work. Although the majority of studies reviewed were largely British and mainly descriptive, evidence from other countries suggests that they are nevertheless applicable to any healthcare programme which is aiming to improve the quality of care.^85–88

It is clear from this review that audit, and thus clinical governance, faces many obstacles. Furthermore, undertaking it at all requires much commitment and strength of purpose from clinicians. Already there appears to be the same confusion and ambivalence about the implications of clinical governance as there was with the introduction of audit.^{89, 90}

The barriers which this review has identified need to be addressed systematically. Some of them have been in place for a long time. Without their solution, clinicians faced with growing and conflicting demands on their clinical workload will fail to prioritise audit and, gradually, attempts to incorporate it into their working day will cease. Without the mechanism of clinical audit it is clear that the concept of clinical governance will perish. Some of the key lessons from the studies published to date are highlighted below.

Foster an environment for audit

Recognition should come from a senior level that audit is a valued activity. The benefits to individuals of conducting audit should be more widely broadcast. Audit can augment both career and professional development. High quality audit will be impeded unless time is provided for audit and mechanisms to provide protected time for audit could be developed. Provision of protected time should be accompanied by a commitment from staff to produce a report and to act on the study findings.

Tackle the problems of multidisciplinary audit

Multidisciplinary audit can be seen as threatening (exposing one's mistakes to others) and as time wasting and ineffective (designing a study by committee). There is some justification for these views, but action is needed to overcome some of these difficulties. In part this will be achieved by fostering a supportive environment for audit. But it may also need staff training in interpersonal skills and in dealing with conflict. The aim should be to demonstrate to staff that the benefits of engaging in multiprofessional audit outweigh the disadvantages.

Emphasise audit facilitation

The key role of the audit facilitator needs greater recognition, and hospital audit committees should have an audit programme for facilitators to implement. Hospitals should assess whether additional audit support staff should be employed to provide hands on help and advice on the design of projects.

Audit facilitators need skills in study design, data collection, computing, and statistical analysis and these areas of expertise should be emphasised when appointing new staff. These training needs of facilitators should therefore be explicitly recognised and resources made available to second facilitators to appropriate courses.

Review staff training programmes

Many of the barriers to successful audit would be overcome by adequate training. The training should emphasise the importance of planning audit carefully and the benefits of conducting a pilot study. As with the audit facilitators, staff training is also needed to deal with conflicts between individuals and between professional groups.

Establish confidentiality of findings

One of the major concerns of staff is that audit may draw attention to deficiencies in care for which they may be held responsible. This legitimate concern should be addressed and mechanisms put in place to ensure confidentiality and to anonymise the findings.

Ensure all relevant staff are involved

One of the keys to successful audit is a strong sense of ownership of the study. If staff identify with a study they are much more likely to accept its findings and, where appropriate, implement change. Thus at the start of any project effort should be made to identify the staff groups who might be interested in the findings. Then a representative from each specialty could be invited to join the audit group.

Establish evaluation programmes

Evaluations of audit programmes such as those undertaken by CASPE need to continue with efforts being focused on the identification of factors at a local level which can promote successful audit. These will help to ensure that the most effective methods are channelled to the most appropriate places. Difficulties such as lack of time, expertise, support, and ineffective information technology systems do not seem insurmountable, but their solution requires more than lip service from those who expect clinicians to undertake audit. Commitment from managers, evidenced by the provision of a structured programme of support and training and the availability of resources to implement realistic changes, may help to persuade those who feel ambivalent and threatened by audit that it can be a useful way of improving the quality of patient care. Without such evidence committed clinicians will fail to sustain their enthusiasm for audit and the more sceptical will not be persuaded that audit is anything more than, at best, a waste of time a professional priority. Clinical audit at its best can be a valuable assistance in delivering such quality. Yet without a coherent strategy aimed at nurturing effective audits, valuable opportunities will be lost. It remains to be seen whether the new ideology of clinical governance can succeed in making audit an accepted and integral part of every clinicians' role. To be effective, clinical governance should pay close attention to the lessons learnt from clinical audit.