Fortnightly Review: How to ensure that guidelines are effective
BMJ 1995; 311 doi: https://doi.org/10.1136/bmj.311.6999.237 (Published 22 July 1995) Cite this as: BMJ 1995;311:237
- Richard Thomson, senior lecturer in public health medicinea,
- Michael Lavender, senior registrar in public health medicineb,
- Rajan Madhok, director of public healthc
- Department of Epidemiology and Public Health, School of Health Care Sciences, Medical School, Newcastle University, Newcastle upon Tyne NE2 4HH,
- North Tyne Health, Newcastle upon Tyne,
- South of Tyne Health Commission, South Shields, Tyne and Wear
- Correspondence to: Dr Thomson.
- Accepted 4 April 1995
Summary points
Summary points
Guidelines are a way to support effective clinical practice
There is a growing body of literature on the factors that influence the effectiveness of guidelines
Reference to these factors will enhance the likelihood of achieving the objectives of guidelines
The use of this series of reflective questions rooted in this literature will support the effective development, dissemination,implementation, and review of guidelines
The assessment of a health practice and the development of a policy for the use of that practice is the final step in a long process that begins with a research idea and ends in actual changes in people's health. All the accumulated research, development, and experience is converted into practical recommendations that largely determine what happens to a patient. In this pivotal position the design of a policy deserves whatever effort is required to ensure that all the proceeding work is put to the best effect1
There is an explosion of interest in guidelines, reflected in a vivid debate ranging from “the best thing since sliced bread” to cries of “cookbook medicine” and fears of constraint on clinical freedom.2 3 4 5 6 7 8 Furthermore, the Department of Health has introduced an initiative to encourage the adoption of evidence based guidelines within purchaser-provider contracts,9 10 and a review of the effectiveness of clinical guidelines has recently been published.11
The subject of guidelines is complex. Even for enthusiasts, ensuring that guidelines are effective by addressing each part of the complex chain of development, dissemination, implementation, and evaluation (fig 1) can be a daunting task. There are an increasing number of publications on the subject--for example, on the details of guideline development and effectiveness.1 12 13 14 15 16 In particular, the seminal work of Grimshaw and Russell has led to a much quoted taxonomy of the factors that influence the effectiveness of guidelines.15 We have drawn together the key messages from this literature and represented them in the form of a series of reflective questions to guide readers through this complex but important maze.
Chain of events to produce effective guidelines
Why guidelines? What are they?
Before addressing the practicalities, it is worth reminding ourselves of the reasons for the prominence of guidelines. These include an emphasis on audit and improving the quality of health care; medical advances and increasingly complex clinical decision making; unexplained variations in clinical practice; heightened public awareness of, and participation in, decision making; and a more explicit debate about the use of limited resources.
Chambers' English Dictionary defines a guideline as “an indicator of a course that should be followed, or of what future policy will be.”17 In clinical work several terms have been used, including guidelines, practice policies, clinical policies, practice parameters, protocols, and algorithms.
Perhaps the clearest definition is that of the Institute of Medicine, guidelines being “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.”18 Guidelines also have a role in supporting quality assurance and audit, including providing the framework against which care can be evaluated.
Regardless of these differing definitions, there are common elements. Guidelines can:
Help patients and professionals to make decisions about health care
Describe appropriate care based on the scientific evidence and broad consensus, leaving room for justifiable variations in practice
Focus on specific circumstances while taking into account organisational factors, community characteristics, and other influences on health care delivery
Act as a focus for quality assessment and improvement activity, including audit.
Development of guidelines
The first step in developing guidelines is choosing the topic or subject (box 1). Choosing a topic simply because there is a desire for guidelines may be inappropriate--the desire may reflect uncertainty due to a lack of empirical evidence to guide practice. If so, no amount of enthusiasm for guidelines will compensate for the lack of evidence. Thus, it may be advisable to start with a topic with a broad consensus of opinion or for which guidelines are already available from another source (see below).
Key questions on choice of topic
Are there large or unexplained variations in practice?
Is the topic important in terms of the process and outcome of patient care?
Is there potential for improvement?
Is the investment of time and money likely to be repaid?
Is the topic likely to hold the interest of team members?
Is consensus likely?
Will change benefit patients?
Can change be implemented?
The development of practice guidelines cannot be successfully undertaken in isolation: it requires a team effort (box 2).19 The effectiveness of guidelines in improving patient care depends not only on the guidelines themselves but also on the nature of the group producing them.20 For example, whether the guidelines have credibility with a peer group of professionals or whether the guidelines gain the support of those who will be responsible for implementing them may critically depend on who is seen to have been active in their development. Moreover, it is important when developing guidelines to consider gaining the participation of those with the power and authority to implement them or to persuade others so to do.
Key questions about guidelines development group
Do we have the right people in the group?
Doctors?
Primary and secondary care?
Nurses?
Therapists?
Managers?
Purchasers?
Patients or patients' representatives?
Researchers?
Can we involve other appropriate people by circulation of drafts or some other form of consultation?
Do we have appropriate skills?
Group leadership?
Group facilitation?
Knowledge of guidelines?
Specialist knowledge of the subject?
Skills in setting standards?
Administrative support?
Skills in reviewing and analysing the literature?
Are there too many or too few of us?
Who will be expected to implement or evaluate the guidelines produced? Have we included them or their representatives? Working of the group
Are we clear about the task for this group?
Can we estimate how long we will need to develop and implement guidelines?
How long will we need to spend on each stage?
Development? (including review of potential sources of others' guidelines)
Dissemination?
Implementation?
Review and evaluation?
Are members of the group prepared to commit sufficient time to the process?
How many meetings will we need?
What length of meeting?
What is our proposed timescale?
Do we need to start from scratch or isit a matter of adopting or adapting others' guidelines?
The group will need skills in the relevant clinical subject, including specialist experience, to ensure that the guidelines produced have professional credibility. The group may need skills in conducting a literature review and in collecting and analysing data. Above all, the group needs a leader with the organisational and communication skills required to take them through the process of guidelines development and on to the stages of implementation and review.
Some members of the group may be unwilling to commit themselves to a process such as this without some idea of the time entailed. At this stage it is well worth taking the trouble to outline a plan for the process, deciding how often to meet and for how long. The plan should include specific objectives. You will then be in a position to estimate the time it will take to complete the process. Don't underestimate the time and effort needed for implementation and evaluation. Give these stages of the process at least the same, if not more, time than you give to the development stage.
Methods
of guideline development
Before starting guidelines development, explore what has already been done elsewhere. It will be unusual to find a subject in which no work on guidelines development has been undertaken. Seek already available guidelines. Review the literature. Talk to colleagues elsewhere. Consider talking to your royal college's audit office or to specialty led clinicians in your hospital, practice, or region. Even if your search identifies only poorly developed guidelines or failed attempts at development, that knowledge will give you a better start than staring at a blank piece of paper and hoping for inspiration, even if only to identify what not to do.
ADOPTION, ADAPTATION, ASSESSMENT
You may have been able to identify guidelines that have already been developed in your chosen subject. If so, you need to decide whether the guidelines can be adopted unchanged or need to be adapted for local use. Adapting guidelines may entail taking broad based guidelines and turning them into locally applicable guidelines or protocols.12 Although ownership is more readily achieved among those who have participated in development, this needs to be weighed up against the benefits in adopting or adapting guidelines already available. Adapting available guidelines to local circumstances may be sufficient to stimulate ownership.
It is important to have some means of assessing the quality of guidelines that have already been produced. For example, the Agency for Health Care Policy Research in the United States has outlined key factors to be considered.21 Others have suggested alternative or complementary strategies,15 which help to identify guidelines that have undergone an appropriate and rigorous process of development. However, these frameworks for assessing guidelines may be statements of the ideal. Such thorough appraisal of the quality of guidelines may be both difficult (because of the lack of documentation on the process of development of available guidelines) and fruitless at present (as most guidelines may not have gone through such a rigorous process of development).22
STARTING FROM SCRATCH
When reference guidelines are not already available you may need to develop them yourselves. There is no single superior method of developing effective guidelines. The method selected will depend on the topic chosen, the experience of the group, the purpose of the guidelines, and the evidence available on the outcomes of clinical practice (box 3). Although ideal approaches have been proposed,1 the application of these strategies are far from ideal, not least because of limited data on health outcomes in many areas of practice. In the meantime, most current examples of guidelines development use a variable combination of “expert” judgment and literature review.
Key questions on development and presentation of guidelines
Have we clearly defined the subject of the guidelines?
Patient group?
Diagnosis?
Investigation?
Treatment or intervention?
Administration?
Do we have access to guidelines already produced on this subject?
Have we approached likely sources of guidance or advice?
Do we need to undertake a literature review?
Computerised bibliographies--for example, Medline on CD ROM
Medical library
Do not forget the “grey” literature
Is the evidence for defining components of the guidelines available to those responsible for their construction?
How shall we take this forward?
Nominal group technique?
Delphi technique?
Consensus conference?
Evidence based model?
What type of guidelines do we want to produce?
Have we clearly defined the end users of the guidelines?
Is the problem or issue clearly defined?
Is our presentation clear and unambiguous?
Do we need to test its clarity?
Is the evidence for our statements presented or readily available?
Is there a rationale for the guidelines?
What is the estimated impact or outcome?
What are the current variations and uncertainities?
If options are available are the decision making factors clear?
How should we present them? Which format will be most appropriate for their proposed use?
Booklet?
Newsletter?
Pocket card?
Posters?
Publication in professional journal?
Computer based?
Other?
Will the chosen medium be sufficiently
durable for its proposed use?
Are the authors acknowledge or stated?
The views of a group of experts may be coalesced through a consensus development process.23 24 25 This method is potentially the most straightforward way of developing guidelines but is limited by the knowledge and experience of the group members, both in the subject and in the various stages of guidelines development. It can be enhanced by preparatory collation, dissemination, and summary of existing research evidence, preferably by a formal structured review of the literature.
Achieving consensus requires some compromise and is not easy. Too often in groups the loudest or most authoritative members have their way, leading to a lack of commitment from the others within the group. However, there are recognised methods of getting the best out of a group such as the nominal group technique26 and the Delphi technique.27
Alternatively, the process of development can be more formal and systematic, and entail using thorough review and analysis of the available literature to produce guidelines based on the evidence of high quality studies.1 21 In this approach, such as the model developed by the Agency for Health Care Policy and Research in the United States, the development group or team is constructed so as to share the load inherent in detailed and systematic literature review and analysis of both clinical evidence and evidence of cost effectiveness. This approach, however, is both costly and time consuming.
There are trade offs between these approaches and limited evidence on which are more effective or cost effective. Local and consensus groups may lack the skills and resources to undertake a synthesis of the evidence in the literature. In this case the validity of the guidelines produced may be compromised, but it is conceivable that the enhanced local ownership, and its effects on implementation, may compensate for this. This approach emphasises the potential for local adaptation of systematically developed national guidelines and is worthy of further research.
What will the product look like?
There is no single format for presenting guidelines. Several presentations can be used, according to such factors as the target group, the intended use and user, and the topic chosen. A combination of text, algorithms, and option lists can all be used so long as the end result is explicit, logical, and unambiguous. Preferably, a rationale should accompany the guidelines, describing the evidence considered by the group and the way in which it came to the decisions contained within the guidelines. The approach of the Agency for Health Care Policy and Research produces summary guidelines, which are supported by detailed manuals containing the evidence collected and synthesised and the rationale behind the statements, but this thoroughness tends to lead to bulky publications.21 28
When the evidence for the statements is strong and professional judgment almost unanimous, this should be reflected in the way the guidelines are expressed. Some components may be mandatory statements, where judgment is not required and variation in decision making is not clinically justified. An example would be the need for admission for observation of a patient with skull fracture after head injury.
The questions in box 3 will help but should be raised at the planning stage.
Dissemination and implementation of guidelines
Dissemination implies bringing the guidelines to the attention of the intended users (box 4). One of the reasons why guidelines fail to bring about change is that they do not reach the intended users. For example, an audit of the distribution of management guidelines on head injury in the Northern region showed that only 63% of the relevant senior house officers had received copies of the guidelines.29
Key questions for dissemination of guidelines
Are we clear about the target users of our guidelines?
How can we best ensure we reach this audience?
What means do we have of reaching the target audience?
Professional bodies?
Specialty representative groups?
Audit networks? Local clinical audit committees or medical audit advisory groups?
Personnel departments and induction packs?
Unit management boards?
District health authorities or family health services authorities
Local medical committees?
Undergraduate and postgraduate lectures?
Conferences and seminars?
Newsletters?
Other?
In what form should we publish and disseminate the guidelines?
In junior doctors' handbooks?
On posters in the relevant departments and wards?
In newsletters or professional news sheets?
As local reports or documents?
In a journal?
In patient literature?
Several of the above?
Other?
Do we need a system of regular dissemination--for example, every six months to junior doctors?
How will we monitor or evaluate dissemination?
What will dissemination cost?
In this process the opinions of potential users of the guidelines may be helpful. A recent survey of general practitioners in South Tees found that publication in a medical journal was considered to be the least preferred option of disseminating guidelines on referral of patients to orthopaedic outpatient clinics.30 General practitioners preferred having guidelines posted to them and being able to discuss them in specifically arranged seminars. These opinions are consistent with the research evidence on guidelines, which shows that publication in a journal is unlikely to support effective implementation.15
Adequate dissemination is necessary for guidelines to be used, but implementation requires strategies to facilitate changes in behaviour (box 5). This can be done in several ways.
Key questions on implementing guidelines
What means can we use to support implementation?
Can medical records or forms be designed to support the guidelines?
Can we provide individual patient feedback? Computerised record with forced choice?
Is there a means of retrospective feedback?
From audit results or case note review?
Should we arrange an educational presentation?
Seminars, lectures, regular discussion and debate?
Are key opinion formers promoting our guidelines?
What are the incentives to implement the guidelines?
The research evidence is that the greater the educational component of dissemination the greater the likelihood of adoption into practice.15 Thus, specific educational interventions--for example, seminars concentrating on the particular guidelines that have been targeted at the end users--are more likely to be effective than including guidelines within the general continuing medical education programme. Simply posting the guidelines or publishing them in professional journals is less valuable in reaching the target audience or in stimulating adoption. Furthermore, the promotion and endorsement by peers, particularly by respected local clinicians, can enhance adoption.
Reminders may support implementation, but the form and timing of reminders may be crucial. They range from patient specific reminders at the time of consultation to general non-specific reminders. Patient specific reminders at the time of consultation could include, for example, attachment of guidelines to the medical record,31 inclusion of guidelines on a desk top computer,32 33 or specially designed clinical records.34 The key factor is that the guidelines are brought to the clinician's attention at the time of the consultation and are thus able to influence the clinical decision prospectively.
The use of guidelines in audit (see below) can increase the likelihood of adoption. Comparative feedback on performance may be a valuable stimulus to change, preferably if linked to the cyclical process of audit with implementation of change and subsequent review.
A review of these various strategies has shown that any of them or combinations of them work to a degree and for varying periods. Strategies specific to individual patients are more likely to encourage the adoption of guidelines and bring about a change in patient care.15
Purchasers and contracting
An important potential lever to support the implementation of guidelines is the purchaser-provider interaction and contracting.9 Purchasers have a legitimate interest in influencing the quality and delivery of care through contracts for services and increasingly through their influence on clinical audit,35 which has been enhanced recently by the transfer of audit funds to purchasers. However, there are problems with this approach that relate to the degree of skills within purchasing organisations, the quality of the interaction with providers, and the methods by which purchasers seek to influence their providers.36 37
Thus, although the opportunities to develop purchasing based on guidelines exist, they may not be as extensive as some have suggested. At the very least, both purchasers and providers need to be cognisant of the factors discussed here that are likely to influence the effectiveness of guidelines. Thus, the imposition of guidelines by purchasers is far less likely to be effective than guidelines that have been developed, adopted, or adapted as a result of shared priority setting and review.
Evaluation
The application of guidelines should produce improved quality of care for patients. However, concentrating on an evaluation of patient outcomes alone as a measure of the success of guidelines is insufficient and may be impractical, given the difficulties of interpreting data on the small numbers of patients in a locality. Thus, the various components of the whole process--that is, development, dissemination, implementation, evaluation, and review--have a part to play (box 6). For example, patient outcome may be unchanged because the target audience has not received the guidelines, because it has not read them, because it has forgotten them, and so on. Without an evaluation of each step in the chain (fig 1), inability to show effectiveness may leave people none the wiser about which link(s) in the chain may have been inadequate.
Key questions on evaluation and revision of guidelines
Evaluation
How will we know if the guidelines are Received?
Read?
Respected?
Used?
Locally evaluated?
Locally promoted or endorsed?
What methods are required to assess each of the above?
Surveys?
Questionnaires?
Case note reviews?
Cyclical criterion based audit?
Routine monitoring?
Do we need advice on methodology?
What gaps in knowledge have been identified through evaluation?
How will we feed back the results of evaluation to those responsible for implementation?
How will appropriate changes in each step of the chain be identified and implemented?
Is there a clear means of evaluation?
Can explicit standards be assessed?
Can locally agreed standards arising from the guidelines be defined?
Can key indicators be selected to give a measure of implementation?
What is the expected outcome and how can it be measured?
Who should assess the guidelines?
Lead clinicians in local audit?
Managers?
External organisations--for example, purchasers?
Professional bodies?
Regional audit structures?
Other?
Review and revision
How often should the guidelines be reviewed or reformulated? What is the likelihood of significant advances in knowledge occurring?
How will their relevance be maintained?
Who will be responsible for initiating review? Who will be needed to undertake a review?
Has evaluation suggested need for review?
How will reviewed guidelines be disseminated to replace redundant versions?
Working within the developing multidisciplinary clinical audit programmes becomes important at this point.38 Audit groups will enhance ownership and enable the dissemination and implementation process to be intimately linked to audit programmes. Furthermore, consideration of the role of audit at an early stage will help to identify which elements of the guidelines are amenable to standard setting and hence be the basis of criterion based audit.39 Evaluation can take the form of comparative evaluation (“Have we improved?”) or absolute evaluation (“Have we achieved a defined standard?”).
Moreover, the audit process may be valuable not only in assessing whether the guidelines have been adhered to (an audit of the process of care) but also in assessing the effectiveness of other links in the chain. Such an approach is applicable to each component of guideline development, dissemination, implementation, evaluation, and review. A standard for dissemination, for example, could be that 100% of the target audience should receive the guidelines. It may not be possible to evaluate all components, but identification of key standards or indicators should be considered.
Updating guidelines
One of the potential criticisms of guidelines is that they could stifle change and innovation.3 This concern can be dealt with by periodic review of the guidelines. Review may be necessary because of advances in medical knowledge, changes in medical practice, changes in local circumstances, or the results of evaluation.
Review is necessary to prevent outdated processes being applied and to incorporate recent research findings. The frequency and detail of review will depend on the subject of the guidelines, including the fluidity of knowledge in the field and local changes in delivery or organisation of services that may require a reappraisal. When planning a scheduled review for the guidelines, any changes will need to be implemented with the same commitment as the original guidelines (box 6).
In addition to the questions specific to each area above there are a series of general questions that will support the development and implementation of guidelines (box 7).
Key questions on sources of support
General
Is there access to developed or published guidelines?
Are there sources of advice and support for guideline development?
Are these international, foreign, national, regional, or local?
International
Are there international organisations or specialty bodies with guideline experience?
Can they help us?
National or regional
Have you approached your royal college or faculty audit office or equivalent?
Have you approached the National Clinical Audit
Information and Dissemination Centre?
Have you approached the Eli Lilly National Medical Audit Centre?
Are there other national or regional specialty bodies worth exploring?
Are there active national or regional patient organisations worth approaching?
Can a member of the regional clinical audit committee advise?
Is there a local university department with skills--for example, a department of epidemiology and public health?
Is there a national or regional database or compendium of good practice in audit or guidelines?
Can the regional college or faculty adviser help?
Is there a regional specialty led clinician and audit group?
Can it help?
Local
Can the postgraduate or medical library help with
A literature search?
Audit and guideline publications?
Specialist and other journals?
Is there a local audit office with information or to provide support?
Are there accessible local audit support staff?
Can the local clinical audit committee or medical audit advisory group help?
Is there a local nursing and therapy audit group?
Conclusions
McKeown states: “Medicine must be prepared to face the tests which are inescapable in private enterprise and which it is almost unique among public activities in having evaded hitherto: Is our work well done? Is it worth doing? and Does it pay its way”40
The process that leads from selecting a topic to having an impact on patient care is complex and full of potential pitfalls. None the less, there is a growing body of research evidence on appropriate approaches. We have reviewed this literature and produced a series of reflective questions to help incorporate this knowledge into practice. We believe that the potential for guidelines to influence the quality of patient care is considerable, particularly when aligned with the process of audit and quality assurance, in the setting of the purchaser-provider interaction, and within an overall culture of continuous quality improvement (fig 2).37
This review arose from work undertaken on behalf of the former Northern Regional Health Authority as part of its strategy to support wider use of guidelines. We produced a guidelines resource pack for a regional guidelines conference; the pack includes overhead masters for the boxes and figures in this article and is available from us on request. We thank the members of the Regional Guidelines Group, particularly Dr Pali Hungin, for their valuable comments.