Objectives: Reliability studies are frequently organized within the context of a large (multicenter) study, with only a small sample of subjects measured by the observers of the large study. To estimate interobserver reliability, data from the large study are not frequently used. In this article, the advantages of combining data from the reliability study and the large study to improve the estimation of intra-class correlation coefficients (ICCs) are highlighted.
Study design and setting: This was done within the scope of estimating fat percentages in the Project On Preterm and Small-for-gestational-age infants-19 (POPS-19) study and with simulations. To calculate ICCs, three approaches were used: (1) the classical approach using data from a reliability study only, (2) the combined variances approach using inter-subject variances from the POPS-19 study, and (3) the maximum likelihood approach using all data.
Results: The ICCs (95% confidence interval [CI]) for fat percentage calculated by the three approaches were 0.84 (0.57, 0.99), 0.94 (0.90, 0.97), and 0.94 (0.88, 0.97), respectively.
Conclusion: The efficient use of data by combining data from a small reliability study with the data from the large study itself for the calculation of ICCs will lead to more precise ICCs.