Article Text

Download PDFPDF

P247 A Simple Guideline Appraisal Instrument Based On IOM Standards
  1. M Mitchell1,2,
  2. B Leas1,2,
  3. J Lavenberg1,2,
  4. D Goldmann1,4,8,
  5. C Umscheid1,2,4,5,6,7
  1. 1Center for Evidence-based Practice, University of Pennsylvania Health System, Philadelphia, USA
  2. 2ECRI Institute–Penn Medicine Evidence-based Practice Center, Philadelphia, USA
  3. 3Clinical Practices of the University of Pennsylvania, Philadelphia, USA
  4. 4Department of Medicine, University of Pennsylvania School of Medicine, Philadelphia, USA
  5. 5Leonard Davis Institute of Health Economics, Philadelphia,USA
  6. 6Center for Clinical Epidemiology and Biostatistics Philadelphia, USA
  7. 7Institute for Translational Medicine and Therapeutics Philadelphia USA Elsevier Health Sciences Philadelphia, USA


Background Scales like AGREE provide a systematic means for appraising guideline quality, but they are lengthy, emphasise methodology over practicality, and are best applied by guideline experts.

Objectives Create a short instrument for guideline appraisal, based on widely accepted standards.

Methods The Institute of Medicine (IOM) identified eight principles that make a guideline ‘trustworthy’. We adapted each principle into an item graded ‘A’, ‘B’, ‘C’, or “NR” (not reported). Guideline assessments are presented as a grid rather than a single score, with each row representing an item, each column a guideline, and cells coloured green, yellow, red or white to reflect the above grades, respectively. Concordance tables mapping AGREE and G-I-N standards to IOM domains were also created.

Results Piloted use of the tool suggests it can distinguish guidelines developed using weak methods and those that are poorly documented. Grids highlight guideline strengths and weaknesses, as well as guidelines that are more trustworthy than their comparators. The concordance table found that AGREE lacks standards for guideline currency and updating, while IOM lacks standards for resource implications.

Discussion Our pilot use of this instrument suggests that while the overall trustworthiness of guidelines is important, using IOM domains to understand sources of guidelines’ weaknesses can help organisations select guidelines best suited for their needs. Further work will examine our instrument’s reliability across users with different levels of expertise.

Implications for Guideline Developers/Users Pilot use of this tool suggests it can be applied by clinicians and administrators who have limited training and time.

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.