Objective To compare three methods of guideline development, to see whether using alternative evidence-based methods resulted in variation of recommendations for treating actinic keratosis.

Methods Method 1 followed a standard multiple session evidence-based approach with a working group. In method 2 recommendations were formulated by a working group during a 2-day conference. Method 3 used one epidemiologist to summarise the evidence and one dermatologist to make clinical recommendations afterwards. Graded recommendations and levels of evidence were compared per therapy across three draft guidelines. The primary outcome was the extent of accordance or discordance. Secondary outcomes were total costs and time period necessary to make a draft guideline.

Results Therapeutic recommendations and levels of evidence differed in some occasions. However, intraclass correlations between levels of evidence were significant (method 1 vs 2: p=0.003; method 1 vs 3: p<0.001). Regarding recommendation variation method 1 and method 2 correlated significant at 0.755 (p=0.001). Method 1 versus 3 and method 2 versus 3 also showed significant, but lower, correlation coefficients (respectively, 0.493 (p=0.026) and 0.673 (p=0.007)). Method 3 was the cheapest and quickest (24 770 euro and 4 months) and method 1 was the most expensive and slowest method (€48 100 euro and 14 months).

Conclusions The value of a guideline using alternative evidence-based methods seems to at least equal that of a guideline composed in multiple sessions, that is, for topics with a monodisciplinary character and a relatively small number of conducted trials. In addition, the presented alternatives were more time- and cost-efficient.