Methods that combine information in the medical literature with expert clinical judgment are needed to determine the appropriateness of use of a procedure. The purpose of this study is to better understand the reliability and construct validity of this process by comparing ratings of appropriateness for diagnostic upper gastrointestinal endoscopy that were developed using different approaches by two independent groups. Both the RAND/UCLA Health Services Utilization Study (HSUS) and the American Society for Gastrointestinal Endoscopy (ASGE) combined scientific data with expert physician judgment to rate the appropriateness of specific clinical indications for the use of upper gastrointestinal endoscopy. This study applies the ratings developed by each group to a nationally representative sample of 1,585 endoscopies performed on people 65 years of age and older in 1981. HSUS developed indications that could be used to rate all 1,585 procedures; ASGE indications were less comprehensive and applied to 70% (n = 1,115) of procedures. Of those rated by both groups, appropriateness category ratings agreed for 94% of the procedures. However, the procedures not rated by ASGE were unevenly distributed across HSUS appropriateness ratings. Twelve percent of procedures rated as appropriate by HSUS were not rated by ASGE, but 80% of procedures rated as equivocal by HSUS and 73% rated as inappropriate by HSUS were not rated by ASGE, for those procedures rated by both approaches there was good agreement; however, a more explicit and comprehensive method may be required if equivocal and inappropriate use of a procedure is to be identified.