Background The National Health Service National Patient Survey Programme systematically gathers patients’ experiences about the care they have recently received. Prioritising quality improvement activities in the accident and emergency (A&E) department requires that survey outcomes are meaningful and reliable. We aimed to determine which method of obtaining summary scores for the A&E department questionnaire optimally combined good interpretability with robust psychometric characteristics.

Methods A&E department questionnaire data from 151 hospital trusts were analysed, covering 49 646 patients. Three methods of grouping and summarising items of the questionnaire were compared: principal components analysis (PCA); Department of Health dimensions; sections according to the patient's journey through the A&E department. The patient-level reliability of summary scores was determined by Cronbach's α coefficients (threshold: α>0.70), construct validity by Pearson's correlation coefficients, and the discriminative capacity by intra-class correlation coefficients (ICCs) and reliability of A&E-level mean scores.

Results The PCA provided the best score reliability on six clear and interpretable composites: waiting time; doctors and nurses; your care and treatment; hygiene; information before discharge; overall. The discriminative power of the concepts was comparable for the three methods, with ICCs between 0.010 and 0.061. A&E sample sizes were adequate to obtain good to excellent reliability of A&E-level mean scores.

Conclusions The A&E department questionnaire is a valid and reliable questionnaire to assess patients’ experiences with the A&E. The discriminative power of six summary scores offers a reliable comparison of healthcare performance between A&Es to increase patient centredness and quality of care.