Background The Korean translated AGREE II instrument has been distrubuted to Korean medical societies in 2011. But inter-rater disagreement issues still existed. The Korean AGREE II scoring guide was therefore developed to reduce inter-rater differences.
Objectives To examine effects of the Korean AGREE II scoring guide on reduction of the inter-rater differences.
Methods Appraisers were randomly assigned to group A or B. The Korean AGREE II instument was provided to all participants, but the Korea AGREE II scoring guide was offered to only group A. Total of 14 appraisers participated and each gudeline was assessed by 8 appraisers. To evaluate the reliability of the Korean AGREE II scoring guide, correlation of scores among appraisers and domain-specific intra-class correlation (ICC) were calculated.
Results Most scores of two groups were comparable. And scoring guide user group showed higher reliability at all guidelines. It showed higher correlation among appraisers and higher ICC values at almost all domains (from 0.272∼0.792 to 0.667∼0.888 in CPG A, from 0.273∼0.682 to 0.424∼0.815 in CPG B).
Discussion Scoring guide reduces the inter-rater disagreement and improves the overall reliability of the Korean-AGREE II instrument.
Implications for Guideline Developers/Users This study identified the performance and utility of the scoring guide as a useful guidance to Korean-AGREE II instrument users.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.