Communication and interpersonal skills are essential elements of a physician's clinical expertise. Since 1998, the interpersonal competencies of over 37,000 internationally-trained physicians have been assessed as part of the Educational Commission for Foreign Medical Graduates (ECFMG) Clinical Skills Assessment (CSA). Standardized patients (SPs) provided ratings of interpersonal skills along 4 dimensions: skills in interviewing and collecting information; skills in counseling and delivering information; rapport; and personal manner. The content of the rating scale, the development and implementation of training materials and procedures, and the psychometric characteristics of the measures are described. Data from over 400,000 simulated patient encounters were analyzed. Correlations with other measures supported the construct validity of the assessment. A generalizability study showed that the ratings were reproducible over encounters. Analysis of individual SP ratings indicated that they were consistent in their application of the scoring rubric. Overall, the findings indicate that SPs, with proper training and a benchmarked scoring rubric, can provide accurate and defensible ratings of physicians' interpersonal skills. These results may also generalize to other clinical skills assessments, or other evaluations that employ raters to judge communication abilities.