Table 4

Within-staff-type ICC and between-staff-type group ICC comparisons of holistic scale score reliability for phases of care and overall score

Reviewer pairs	No. of reviewer pairs (or triplets)	No. of case notes		Weighted mean ICC^* between ranked scores
Reviewer pairs	No. of reviewer pairs (or triplets)	No. of case notes		Admission/investigations and examination phase	Initial management phase	Predischarge phase	Overall care
Within-staff-type ICC results
Doctor vs doctor	4	127	Weighted mean	0.58	0.70	0.46	0.52
Doctor vs doctor	4	127	95% CI	0.48 to 0.68	0.63 to 0.78	0.34 to 0.59	0.41 to 0.62
Nurse/clinical vs nurse/clinical	4	144	Weighted mean	0.50	0.22	0.43	0.46
Nurse/clinical vs nurse/clinical	4	144	95% CI	0.38 to 0.62	0.07 to 0.37	0.30 to 0.55	0.34 to 0.59
Non-clinical audit staff vs non-clinical audit staff	2	87	Weighted mean	0.35	0.10	0.39	0.47
Non-clinical audit staff vs non-clinical audit staff	2	87	95% CI	0.16 to 0.54	−0.10 to 0.30	0.21 to 0.57	0.22 to 0.66
Between staff type comparisons of ICC
Doctor vs nurse/clinical	5	179	Weighted mean	0.23	0.25	0.29	0.43
Doctor vs nurse/clinical	5	179	95% CI	0.09 to 0.37	0.12 to 0.39	0.16 to 0.43	0.31 to 0.54
Doctor vs non-clinical audit staff	6	188	Weighted mean	−0.01	0.03	0.25	0.24
Doctor vs non-clinical audit staff	6	188	95% CI	−0.15 to 0.12	−0.11 to 0.16	0.12 to 0.38	0.12 to 0.37
Nurse/clinical vs non-clinical audit staff	1	34	Weighted mean	−0.12	0.19	0.47	0.43
Nurse/clinical vs non-clinical audit staff	1	34	95% CI	−0.44 to 0.23	−0.15 to 0.49	0.17 to 0.70	0.11 to 0.67

ICC, intraclass correlation.
↵* Weighted mean ICC: estimates from the different combinations of reviewers were pooled using a weighting that was inversely proportional to the variance of the estimate.