Article Text

Download PDFPDF
The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies
  1. Pablo Alonso-Coello1,2,
  2. Affan Irfan3,
  3. Ivan Solà1,
  4. Ignasi Gich1,2,
  5. Mario Delgado-Noguera4,5,
  6. David Rigau1,
  7. Sera Tort1,
  8. Xavier Bonfill1,2,
  9. Jako Burgers6,7,
  10. Holger Schunemann8
  1. 1Iberoamerican Cochrane Centre. Clinical Epidemiology and Public Health Department. Institute of Biomedical Research (IIB Sant Pau), Spain
  2. 2CIBER de Epidemiología y Salud Pública (CIBERESP), Spain
  3. 3Interactive Research and Development, The Indus Hospital, Korangi Crossing, Karachi, Pakistan
  4. 4Department of Pediatrics, University of Cauca, Popayán, Colombia
  5. 5Pediatrics, Obstetrics and Gynecology and Preventive Medicine Department, Universidad Autónoma de Barcelona, Barcelona, Spain
  6. 6Dutch Institute for Healthcare Improvement CBO, Utrecht, The Netherlands
  7. 7Scientific Institute for Quality of Healthcare (IQ healthcare), Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands
  8. 8Department of Clinical Epidemiology & Biostatistics, McMaster University Health Sciences Centre, Hamilton, Canada
  1. Correspondence to Dr Pablo Alonso-Coello, Iberoamerican Cochrane Centre. Clinical Epidemiology and Public Health Department. Institute of Biomedical Research (IIB Sant Pau), Barcelona, Spain; palonso{at}


Background Despite the increasing number of manuals on how to develop clinical practice guidelines (CPGs) there remain concerns about their quality. The aim of this study was to review the quality of CPGs across a wide range of healthcare topics published since 1980.

Methods The authors conducted a literature search in MEDLINE to identify publications assessing the quality of CPGs with the Appraisal of Guidelines, Research and Evaluation (AGREE) instrument. For the included guidelines in each study, the authors gathered data about the year of publication, institution, country, healthcare topic, AGREE score per domain and overall assessment.

Results In total, 42 reviews were selected, including a total of 626 guidelines, published between 1980 and 2007, with a median of 25 CPGs. The mean scores were acceptable for the domain ‘Scope and purpose’ (64%; 95% CI 61.9 to 66.4) and ‘Clarity and presentation’ (60%; 95% CI 57.9 to 61.9), moderate for domain ‘Rigour of development’ (43%; 95% CI 41.0 to 45.2), and low for the other domains (‘Stakeholder involvement’ 35%; 95% CI 33.9 to 37.5, ‘Editorial independence’ 30%; 95% CI 27.9 to 32.3, and ‘Applicability’ 22%; 95% CI 20.4 to 23.9). From those guidelines that included an overall assessment, 62% (168/270) were recommended or recommended with provisos. There was a significant improvement over time for all domains, except for ‘Editorial independence.’

Conclusions This review shows that despite some increase in quality of CPGs over time, the quality scores as measured with the AGREE Instrument have remained moderate to low over the last two decades. This finding urges guideline developers to continue improving the quality of their products. International collaboration could help increasing the efficiency of the process.

  • Clinical Practice Guidelines (CPG)
  • AGREE instrument
  • appraisal
  • quality
  • healthcare quality

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


  • Competing interests HS is a cochair of the GRADE Working Group, an international collaboration of methodologists, epidemiologists, clinicians and guideline developers, and supports the work of GRADE internationally. He receives no direct personal payments for work as a member of the GRADE Working Group. JB is a member of the AGREE Reseach Trust (ART) that aims to facilitate the distribution, maintenance and improvement of the AGREE Instrument and to encourage its development through collaborative research projects. ART is a charitable trust that does not receive any income. PA-C is a member of the GRADE Working Group. He receives no direct personal payments for work as a member of this group.

  • Provenance and peer review Not commissioned; externally peer reviewed.