Background Clinical practice guidelines are an important component of efforts to improve quality of care and rationalise the introduction of new medical technologies. As evidence-based medicine and comparative effectiveness research become more prominent, the rigour and transparency of guideline development is becoming increasingly important.

Context Since 1988, the American Academy of Ophthalmology has published ophthalmic practice guidelines known as Preferred Practice Patterns (PPPs). Over time, the Academy has introduced increasingly rigorous processes for grading the evidence underpinning the PPPs.

Description of Best Practice Prior to 2000, the PPPs were effectively consensus-based, with no formal processes for identifying or synthesising evidence, and no system for grading evidence quality. In 2000, a three-level system was introduced to denote the quality of the evidence supporting PPP recommendations, with Level I denoting strongly evidence-based statements and Level III denoting statements unsupported by evidence. In 2001, the level definitions were revised to denote the quality of specific study designs, and explicit guidance for synthesising bodies of evidence was developed. A third revision in 2006 tightened the criteria for Levels I and II, and created a Level IV to differentiate poor-quality studies from expert opinion. In 2011, the levels were revamped to be consistent with SIGN and GRADE methods.

Lessons The Academy has worked for years to make its evidence grading systems more rigorous. This has included using a level-based system to clearly link PPP recommendations to evidence quality, revising the level definitions for greater clarity, and providing explicit direction on evidence synthesis.