Elsevier

The Lancet

Volume 355, Issue 9198, 8 January 2000, Pages 103-106
The Lancet

Articles
Practice guidelines developed by specialty societies: the need for a critical appraisal

https://doi.org/10.1016/S0140-6736(99)02171-6Get rights and content

Summary

Background

There is increasing concern about the quality, reliability, and independence of practice guidelines. Because no information is available on the methodological quality of the guidelines developed by specialty societies, we undertook a survey on those published in peer-reviewed journals.

Methods

Practice guidelines produced by specialty societies and published in English between January, 1988, and July, 1998, where identified through MEDLINE. Their quality was assessed in terms of whether they reported: the type of professionals and stakeholders involved in the development process; the strategy to identify primary evidence; and an explicit grading of recommendations according to the quality of supporting evidence.

Findings

Overall, 431 guidelines were eligible for the study. Most did not meet the criteria: 67% did not report any description of the type of stakeholders, 88% gave no information on searches for published studies, and 82% did not give any explicit grading of the strength of recommendations. There was improvement over time for searches (from 2% to 18%, p<0·001) and explicit grading of evidence (from 6% to 27%, p<0·001). All three criteria for quality were met in only 22 (5%) guidelines.

Interpretation

Despite improvement over time, the quality of practice guidelines developed by specialty societies is unsatisfactory. Explicit methodological criteria for the production of guidelines shared among public agencies, scientific societies, and patients' associations need to be set up. Common standards of reporting, following the same principles that led to the CONSORT statement for randomised clinical trials, should be promoted.

Introduction

Over the past 20 years practice guidelines have become an increasingly popular tool for synthesis of clinical information so as to change clinical practice and improve quality of health care. Medical specialty societies have been particularly active in producing such guidelines together with agencies whose remit includes technology assessment and health care evaluation.

Such a quantitative growth in the number of guidelines available in different specialties is, however, a source of concern since there is evidence that recommendations produced by different groups can be conflicting1, 2 and some researchers go so far as to say they are invalid, unreliable, and irrelevant.3

Thus, growth in the numbers of guidelines without application of rigorous criteria for their production4, 5 could undermine their credibility and lead to harm to the patient if the wrong recommendations were put into practice.1, 2

To see whether these concerns about the quality of existing guidelines have any foundation, we undertook a survey of practice guidelines officially issued by specialty societies over the past 10 years.

Section snippets

Methods

Practice guidelines developed by specialty societies were identified through MEDLINE, from January, 1988, to July, 1998. The search strategy is outlined in panel 1. The list of references identified through this process was examined by three independent assessors from our team, and papers and documents including in the title word such as “guidelines”, “parameters”, “standards”, “consensus”, explicitly written by a specialty society (or on behalf of a specialty society), were eligible as long as

Results

Overall, of 3129 abstracts and titles retrieved from the MEDLINE search, 576 (18%) papers were identified as potentially eligible. 145 (25%) were excluded after closer scrutiny of the published reports because: 90 were narrative or systematic reviews or position papers; 23 were duplicate publications of the same guideline or short summaries of full reports published elsewhere; and 32 were documents about professional curricula or organisational standards.

A total of 431 guidelines were eligible

Discussion

Our survey shows that the quality of reporting of practice guidelines produced by specialty societies fell short of acceptable methodology up to mid 1998. If practice guidelines are to be widely accepted as an improvement tool for quality, greater attention needs to be paid to the methods used to develop them.9, 10, 11, 12

In the USA the Institute of Medicine's reference definition of practice guidelines appropriately underscores that they are “systematically developed statements”,11 thus

References (20)

  • R Thomson et al.

    Guidelines on anticoagulant treatment in atrial fibrillation in Great Britain: variation in content and implications for treatment

    BMJ

    (1998)
  • N Unwin et al.

    Implications of applying widely accepted cholesterol screening and management guidelines to British adult population: cross-sectional study of cardiovascular disease and risk factors

    BMJ

    (1998)
  • H Varonen et al.

    Practice guidelines in Finland: availability and quality

    Qual Health Care

    (1997)
  • A Hibble et al.

    Guidelines in general practice: the new Tower of Babel?

    BMJ

    (1998)
  • G Feder

    Guidelines for clinical guidelines

    BMJ

    (1998)
  • MK Murphy et al.

    Consensus development methods and their use in clinical guideline development

    Health Technol Assessment

    (1998)
  • K Dickersin et al.

    Identifying relevant studies for systematic reviews

    BMJ

    (1994)
  • BJ Smith et al.

    Modern methods of searching the medical literature

    Med J Aust

    (1992)
  • J Grimshaw et al.

    Achieving health gain through clinical guidelines: I—developing scientifically valid guidelines

    Qual Health Care

    (1993)
  • F Cluzeau et al.

    Development and application of a generic methodology to assess the quality of clinical guidelines

    Int J Qual Health Care

    (1999)
There are more references available in the full text version of this article.

Cited by (0)

View full text