Background Clinical practice guidelines should provide a rational basis for healthcare decisions; however, their quality is often poor.

Objectives To systematically identify and describe guideline appraisal tools and draw conclusions for guideline appraisal.

Methods We conducted a systematic search in MEDLINE, EMBASE and the Cochrane Library for English and German-language guideline appraisal tools published after 1995. Reference lists of included publications were also screened. Dimensions of guideline quality were then generated from these publications and from articles by Cluzeau 1999, Graham 2000 and Vlayen 2005. Finally, the questions contained in the appraisal tools were allocated to the quality dimensions and summarised.

Results Overall, 40 appraisal tools were included and 13 quality dimensions identified. The main focus was the identification, assessment and presentation of evidence in guidelines. Questions on dealing with norms and values in guideline development, patient involvement, conflicts of interest, or implementation of guidelines into clinical practice were rare. The tools often assessed the appropriate documentation of the guideline development process (e.g. reporting of the search strategy), without addressing the appraisal of content of the development process (e.g. appropriateness of the search strategy) and the appraisal of clinical content.

Discussion Because many appraisal tools do not contain questions on norms and values or on potential conflicts of interest of guideline authors, important aspects potentially influencing the reliability of guidelines are not covered. In addition, an appraisal of content of the guideline development process and an appraisal of clinical content are often lacking.