Background Health technology assessment (HTA) within the context of a managed care organisation requires rigorous evidence assessment completed in a timely and efficient manner. A modified version of the GRADE evidence grading system was piloted in a long-standing HTA programme to evaluate new and existing medical technologies with high-cost, high-volume impact.

Context An HTA programme developing evidence-based guidance to inform technology acquisition and implementation strategies within a large, US-based healthcare system.

Description of Best Practice A modified GRADE approach was used to assess the quality of evidence for six health technologies over a 6-month period. Although the application of GRADE required additional analytical time, evidence assessments were completed within a reasonable timeframe. Application of the GRADE framework allowed technology committee members to more easily understand the quality of a body of evidence, weigh the benefits and harms, account for patient values, and assess potential resource and operational implications.

Lesson for Guideline Developers Prior to adopting GRADE, many new technologies were assessed as having “insufficient” evidence. The GRADE approach provided greater clarity, and evidence that would previously have been classified as “insufficient” was graded as either “low” or “very low” quality, allowing for greater flexibility and transparency in decision-making when moving from evidence to recommendations. Diagnostic and prognostic tests or devices continued to present unique challenges as well as technologies for which limited comparative evidence was available, and more guidance for in these areas is needed. Additional elaboration on resource and operational concerns specific to evaluating new technologies would be useful to HTA programmes.