• clinical practice guidelines
• guideline recommendations

Most people who pick up a clinical practice guideline will only ever read the recommendations. Even though guidelines are an efficient exercise in research synthesis and summary, the full documentation can run to hundreds of pages—a volume that is beyond all but the most committed reader. In an attempt to address this, guideline programmes produce summary versions. These are usually formed solely of the guideline recommendations and presenting them in this way removes the possibility of the reader checking the validity of any or all of the recommendations. This means that guideline developers, and the programmes that they work with, must ensure that their processes produce recommendations that accurately reflect, not only the content of the scientific evidence, but also the appropriate range of clinical factors. In this issue of QSHC Manna et al¹ present an analysis of the degree to which guideline recommendations reflect clinically important facets of ethnicity and conclude that they do so to a variable extent.

The content of guideline recommendations is influenced by a range of factors. An assessment of the nature of the published evidence (accepting that some may come from the unpublished “grey” literature) addresses the methodological quality of the evidence in terms of threats to its validity. Assessing the applicability of the evidence to the population of interest deals with the issues of generalisability—the extent to which the subjects enrolled into studies reflect the population with which the guideline will be used. The criteria for making judgements on the nature of the published evidence are best developed for studies examining a cause and effect relationship (effectiveness studies) and are reflected in many of the evidence and recommendation grading systems currently in use by guideline development agencies. Epidemiological studies are an important component of any guideline as they are the evidence that will characterise the range and complexity of care that has to be addressed within it. They will identify the important characteristics of individuals that should be reflected both in the inclusion and exclusion criteria of the effectiveness studies and the content of the recommendations. However, unlike effectiveness studies, they are one of the areas of a guideline that are more likely to draw on a national, as opposed to international, literature. Furthermore, epidemiological studies represent a challenge to guideline methodologists to develop the evidence grading systems which appropriately reflect the important dimensions of validity and avoid the current situation of their automatic downgrading by systems designed to grade effectiveness studies.

Aside from the published evidence, three other factors will influence the content of a recommendation. The first of these is costs. Assuming that issues of harm, a potential cost, will be dealt with from the published evidence, then costs usually encompass two considerations: (1) the relative costs of competing technologies set against their effects, and (2) the likely cost impact of the guideline viewed in the context of the overall budget available for health care. This dimension of the guideline depends crucially on how cost and effect are balanced, the quality of the primary data contributing to this, and the nature and scale of the assumptions that have to be made.²

The two other factors that influence the content of recommendations are interlinked, less tangible, and not usually defined. They both refer to the members of the guideline development group and are their knowledge of the healthcare system and their general and healthcare specific beliefs and values. The guideline development group is responsible for interpreting the published evidence in the light of their knowledge and experience. This provides an important reality check on the content of recommendations, making it more likely that feasible recommendations are written. It also allows the group to check that the guideline is appropriately addressing healthcare issues for all of the relevant population. These functions mean that the composition of the guideline development group is of fundamental importance. As Shekelle et al³ observed: “There is good evidence that, when presented with the same evidence, a single specialty group will reach different conclusions than a multidisciplinary group, with the former being systematically biased in favor of performing procedures in which the specialty has a vested interest.^4,5 For example, the conclusions of a group of vascular surgeons favored the use of carotid endarterectomy more than did a mixed group of surgeons and medical specialists⁶.” An international collaborative group, the GRADE group, has reviewed all of the current major guideline grading systems and is currently piloting a new system aimed at making both the treatment of evidence and value judgements more explicit and the decision making in deriving recommendations more transparent.

Value judgements about content are also made at another point in the process—when the original scope of the guideline is defined. Although a guideline development group can argue to vary the content of a guideline, they will usually have the parameters of the guideline set for them by some sort of formal process. For instance, the National Institute for Clinical Excellence (NICE) in England and Wales has a formal process for defining the content of a guideline before starting its development. Here the reality check includes a process of external consultation with those who have an interest in the guideline.

Increasing the sophistication with which guidelines address the validity of their recommendations will not, however, be without cost. The more detailed and specific recommendations are, the longer the guideline will be. Within the context of a full version of a guideline this will not necessarily make a discernable difference as the recommendations form only a small part of the whole. Where the difference will be noticeable is in the summary versions. In the recent “short form” version of the guideline on schizophrenia produced by NICE,⁷ 20 of the 64 pages are given over solely to the recommendations. Further specification within the recommendations could easily double this length. Such summaries are already way beyond a “single side of A4”, the beloved utopia of guideline development. If the product of greater validity and further sophistication is more detailed (and thus longer) recommendations, there will need to be a corresponding increase in the sophistication of the thinking and methods of those responsible for implementing them. Given that the short form guideline on schizophrenia produced by NICE also contains a corresponding 19 page representation of the guideline for “people with schizophrenia, their advocates and carers, and the public”, we can be sure that reduction of the guideline to a single sheet for clinicians is not an appropriate response.

Having been handed a scope, guideline development groups will always struggle with interpreting the published evidence. Given the potential for the guideline content to be influenced by the range of factors described, it is too late to leave consideration of this to the point when recommendations are being written. If the aim is to produce a guideline that appropriately addresses the needs of all of a population (including those determined by ethnicity¹), the implications of this for writing the recommendations should be one of the first—not the last—things to consider. However, such improvements will present challenges to all involved in developing, implementing, and using the guidelines that result.