Objective: The objective of this study was to evaluate the convergent validity of 3 types of utility measures: standard gamble, time tradeoff, and rating scale.
Research design: A prospective cohort of 120 men with advanced prostate cancer were first asked to rank order 8 health states, and then utility values were obtained from each participant for each of the 8 health states through 2 of the 3 techniques evaluated (standard gamble, time tradeoff and rating scale). Participants were randomly assigned to 1 of 3 possible pairs of techniques. The validity of the 3 methods, as measured by the convergence and raw score differences of the techniques, was assessed with ANOVA. The ability of the techniques to differentiate health states was determined. The inconsistencies between rankings and utility values were also measured. Proportions of illogical utility responses were assessed as the percent of times when states with more symptoms were given higher or equal utility values than states with fewer symptoms.
Results: There were significant differences in raw scores between techniques, but the values were correlated across health states. Utility values were often inconsistent with the rank order of health states. In addition, utility assessment did not differentiate the health states as well as the rank order. Furthermore, utility values were often illogical in that states with more symptoms received equal or higher utility values than states with fewer symptoms.
Conclusions: Use of the utility techniques in cost-effectiveness analysis and decision making has been widely recommended. The results of this study raise serious questions as to the validity and usefulness of the measures.