аЯрЁБс > ўџ J L ўџџџ I џџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџџьЅС '` №П ХI bjbj$$ .L F№ F№ 5. џџ џџ џџ Є Р Р Р Р , ь м ь p | | | | | | | ! ! ! ! ! ! $ Ш h 0 Є E Q _ | | _ _ E | | я
я
я
_ т | | _
Р я
_ я
я
я
| @ІЄ8ЩNЭ Р A
^ я
K
Ќ 0 м я
д
F д я
д я
\ | " я
Ж Ъ | | | E E х
| | | м _ _ _ _ Є М М џџџџ APPENDIX B Details of statistical methods
Construction of composites
Composite item sets were readily available for the latter two of the three approaches outlined above (background), but analysis was undertaken to identify a set of items based on the factor analysis approach. Exploratory factor analysis (EFA) and Principal Components Analysis (PCA) are two approaches to locate dimensions of a data set. Simplistically, EFA derives a mathematical model from which factors are estimated, whereas PCA merely composes the original data into a set of linear combinations. ADDIN REFMGR.CITE Field A200575Discovering statistics using SPSSBook, Whole75Discovering statistics using SPSSField A2005Not in File2LondonSage publications21 It is beyond the scope of this study to discuss the approaches in detail but, in general, solutions differ little. The analysis was performed using PCA, with variance maximizing rotation (varimax). The goal of the rotation is to get a clear pattern of the loadings: that is, variables that are clearly marked by high loadings on one factor and low loadings on others. Varimax tries to load a smaller number of variables highly onto each factor resulting in more interpretable clusters of factors, which are uncorrelated with one another. In PCA several criteria need to be fulfilled. The Kaiser-Meyer-Olkin Measure of Adequacy (KMO) is a measure of sampling adequacy (threshold: KMO >0.60). Bartlett's test of sphericity is used to test the null hypothesis that the variables in the population correlation matrix are uncorrelated (threshold: p<0.05). The Eigenvalue represents the amount of the total variance explained by the factor (threshold: Eigenvalue > 1, also known as the Kaiser criterion). Multiple analyses were performed. In case an item loaded on more than one factor, the factor with the highest loading was used. In a subsequent step, factor loadings were obtained (threshold: factor load > 0.40) for the factor composites. Internal consistency was calculated; if Cronbachs alpha ( Б) i n c r e a s e d i f a n i t e m w a s l e f t o u t o f t h e f a c t o r , t h e i t e m w a s d r o p p e d . T h e f a c t o r s t r u c t u r e i n t h e f i n a l P C A f u l f i l l e d t h e s t a t i s t i c a l c r i t e r i a . N e v e r t h e l e s s , t o i m p r o v e t h e c l a r i t y a n d i n t e r p r e t a t i o n o f t h e f a c t o r s , w e d e c i d e d t o b r e a k d o w n a l a r g e f a ctor that covered multiple quality aspects into three factors, each measuring a single quality aspect.
Summary scores were calculated as the means of the experience scores for the items contributing to the composite after PCA, Department of Health dimension and questionnaire section. Respondents who answered less than half of the items in a concept did not contribute to the mean scale score. The section arrival at the ED was left out of the analysis. This section had too few cases left after the screener question How did you travel to the hospital?.
The concepts and items of the three different methods to reduce the data are presented in table 1. The codes underneath each composite, dimension, or section refer to the number of the item in the questionnaire. Cronbachs alpha for internal consistency ( Б) w a s c a l c u l a t e d t o e s t i m a t e t h e r e l i a b i l i t y o f t h e t h r e e c o n c e p t s . T h e s e c t i o n t e s t s c o n s i s t e d o f o n e i t e m , t h e r e f o r e C r o n b a c h s a l p h a c o e f f i c i e n t c o u l d n o t b e c a l c u l a t e d . C r o n b a c h s a l p h a c o e f f i c i e n t s a b o v e 0 . 7 0 w e r e r e g a r d e d a s r e l i a b l e . A D D I N R E F M GR.CITE Frost200774What is sufficient evidence for the reliability and validity of patient-reported outcome measures?Journal74What is sufficient evidence for the reliability and validity of patient-reported outcome measures?Frost,M.H.Reeve,B.B.Liepa,A.M.Stauffer,J.W.Hays,R.D.2007/11Clinical Trials as TopicData Interpretation,StatisticalHumansmethodsPatient SatisfactionProduct LabelingPsychometricsQualitative ResearchReproducibility of ResultsResearch Designstandardsstatistics & numerical dataTreatment OutcomeUnited StatesValidation Studies as TopicNot in FileS94S105Value.Health10 Suppl 2Women's Cancer Program, Mayo Clinic, Rochester, MN, USAPM:17995479Value.Health12 Construct validity was studied by calculating Pearsons correlation coefficients between the concept scores (Table 2). Pearsons correlation coefficient expresses the similarities of underlying constructs of the concepts. A correlation above 0.70 indicated that both concepts partially measured the same construct. Since the relationship between the concepts is partly obscured by random error unattenuated correlations were calculated as well. Table 3 shows the estimated values such as mean, standard deviation and correlation. Additionally, the variance per A&E department and the intra-class correlation coefficient (ICC) were calculated. The variance describes the variability of the A&Es, whilst the ICC expresses the discriminative power of the concepts. The discriminative power is a general assessment of differences between healthcare providers; the variance attributable to providers can be tested for significance. The magnitude of the variance between providers may then be expressed as a proportion of the total variance on a scale from 0-1. ADDIN REFMGR.CITE Boer201177The discriminative power of patient experience surveysJournal77The discriminative power of patient experience surveysBoer,D.D.Delnoij,D.Rademakers,J.2011/12/6PatientsNot in File332BMC.Health Serv.Res.111PM:22145965BMC.Health Serv.Res.13
Next, the estimates were repeated after adjusting the data for age (8 categories) and gender of the respondents ADDIN REFMGR.CITE Healthcare Commission2005137Variations in the experiences of patients in England: Analysis of the Healthcare Commission's 2003/2004 national survey of patientsReport137Variations in the experiences of patients in England: Analysis of the Healthcare Commission's 2003/2004 national survey of patientsHealthcare Commission2005PatientsEnglandanalysisNot in File244 and again, after creating a more homogenous sample. The effect of heterogeneity of the A&Es was investigated with a more homogenous sample, which constructed by deleting from the original sample all trusts characterized as multiservice, specialized or unknown. Decreases of variances and related statistical measures imply that differences between trusts are partially caused by their characteristics.
Finally, A&E-level reliability, which express the proportion of variation in A&E-level mean scores attributable to true variation between A&Es, was estimated. A&E-level reliability was calculated using generalizability theory. ADDIN REFMGR.CITE Shavelson RJ1989348Generalizability theoryJournal348Generalizability theoryShavelson RJWebb NMRowly GL1989/6/1Not in File922932American Psychologist446American Psychologist1Streiner DL2008350Generalizability theoryBook Chapter350Generalizability theoryStreiner DLNorman GR2008Not in File211246FourthHealth measurement scales; a practical guide to their development and use9OxfordOxford University Press35,6 The essence of generalizability theory is the recognition that in any measurement situation there are multiple sources of error variance, due, for instance, to random sampling. The theory contains two stages. In the first stage, called G-study, the variances are used to create G-coefficients, indications of the classical reliability coefficient, given the actual sample size. The G-coefficients look at the proportion of total variance due to the object of measurement. In the final step the variances derived from the G-study are used to set the sample sizes to obtain a reliability of 0.7, 0.8 or 0.9. This is called a D-study.
All analyses were performed using the statistical software SPSS 19.0 and R 2.10.1.
ADDIN REFMGR.REFLIST REFERENCES
1. Field A. Discovering statistics using SPSS, 2 edition. London: Sage publications, 2005.
2. Frost MH, Reeve BB, Liepa AM, Stauffer JW, Hays RD. What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value. Health 2007; 10 Suppl 2: S94-S105.
3. Boer DD, Delnoij D, Rademakers J. The discriminative power of patient experience surveys. BMC. Health Serv. Res. 2011; 11: 332.
+ , - H Ѕ І Ќ Й =
>
R S T U І Ї p s v Ђ Ю б Љ * : = E ѓъѓсиЬРДРДРЃЃЃРwРldldldldldldYld h\| h~./ mH sH h~./ mH sH hЂ h~./ mH sH hбBв h~./ B*mH ph
sH hWС hWС B*H*mH ph
sH hdPќ B*mH ph
sH j hWС B*UmH ph
sH hWЙ B*mH ph
sH h~./ B*mH ph
sH hфv h~./ 5mH sH h%uM 5mH sH hЊ[ 5mH sH hA 5mH sH h%uM h%uM 5mH sH " , - H h i 3 Г' Б, Й5
6 6 16 26 6 U7 к7 ЉH I ТI УI § § ѕ ѕ ѕ ѕ ѕ ѕ э х х р и С С С С С Ќ Ќ
Ц h ф§dр ^`ф§gddPќ
Ц h ф§dр Є№ ^`ф§gddPќ $a$gddPќ gd"70 dh gd"70 dh gdмdC dh gd~./ ХI ў E L M ] _ i Ї Н б в О к р } Ў Б а х њ 5 7 8 < T W Y ` x , l z Ј Њ Ќ Ў А О ш ъ X Y Z [ \ ѕэѕэсвсвсѕэѕэѕэсѕэѕэѕэЧэЧэЧэЧэЧэЧэЧэЧэЧэЧэЧэѕэѕэѕэѕэѕэѕэЛГЛЇЛэ hWС hWС H*mH sH hdPќ mH sH j h~./ UmH sH h}V h~./ mH sH hМ_ы h~./ B*mH ph
sH h~./ B*mH ph
sH h~./ mH sH hЂ h~./ mH sH >\ Й С Ч б г м щ ѕ ! (! )! :! H! O! [! \! g! z! ! ! ! ! ! ! " " " " +" 4" ;" C" |" " Љ" Њ" Э" п" р" ## *# ,# 4# V# R$ S$ {$ |$ }$ Ў' Џ' А' Б' Г' "( #( + + ѕътътътътътътътътътътътътътзтътътътътъЯътътътътътътУЛУЏУтЇЛ j hмdC UmH sH hмdC mH sH hWС hWС H*mH sH hdPќ mH sH j h~./ UmH sH hWС mH sH hzk h~./ mH sH h~./ mH sH hЙ h~./ mH sH hЁ
ј h~./ mH sH =+ + + Љ+ Ў+ - - 93 :3 =3 >3 №3 ё3 f4 q4 4 К4 q5 5 5 5 Й5 њ5 ћ5 §5 6
6 6 6 %6 &6 '6 ѓчпзпЫУЫЗЫпЌпЄпЄпппxjajU h"70 hdPќ 5mH sH hWС 5mH sH j hWС 5UmH sH h"70 5mH sH h"70 mH sH hмdC mH sH hд@ hмdC mH sH hWЙ mH sH hЕ8l mH sH hЙ hмdC mH sH hdPќ hdPќ H*mH sH hdPќ mH sH j hdPќ UmH sH h4ю mH sH hмdC mH sH j hмdC UmH sH hWС hWС H*mH sH '6 06 16 g6 k6 к7 H H ЉH ТI УI ФI ХI іъпзпепЭпФЖЊ h~./ h%uM 5mH sH j hWС 5UmH sH hdPќ 5mH sH hdPќ mH sH Uh"70 mH sH hdPќ hdPќ mH sH h"70 hdPќ 5mH sH hA 5mH sH 4. Healthcare Commission. Variations in the experiences of patients in England: Analysis of the Healthcare Commission's 2003/2004 national survey of patients. 2005.
5. Shavelson RJ, Webb NM, Rowly GL. Generalizability theory. American Psychologist 1989; 44: 922-932.
6. Streiner DL, Norman GR. Generalizability theory. Health measurement scales; a practical guide to their development and use. Oxford: Oxford University Press, 2008; 211-246.
УI ХI ъ
Ц h ф§dр ^`ф§gddPќ , 1hА. АЦA!А"А#$%А АФАФФ F @ёџ F S t a n d a a r d CJ _HaJ mHsHtH L A ђџЁ L S t a n d a a r d a l i n e a - l e t t e r t y p e V i ѓџГ V S t a n d a a r d t a b e l і 4ж
l 4ж aі . k єџС .
G e e n l i j s t F ђ F љ B a l l o n t e k s t CJ OJ QJ ^J aJ 5. L џџџџ , - H Э
Ю
I G! O* Ѓ* Є* Ч* Ш* %+ ы+ p, - - 2. 3. 7. 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 H Э
Ю
I G! O* 7. Њб 00 Њб 00 Њб 00 Њб 00 Ђ 00 @ 0 @ 0
0 0 E \ + '6 ХI ! " УI ХI % ХI = R T ю № D F И Џ Б )" Я' г' Є* Л* 3. 5. QџQTџQTџQџQџQџџџ )U фкх *U Dд 7. 7. 8 *urn:schemas-microsoft-com:office:smarttagsCity 9 *urn:schemas-microsoft-com:office:smarttagsplace У Ц Э Џ Д & З T ^ l
v
G Q | >+ C+ и+ н+ љ+ , , , - '- 5- :- - - 7. б' г' ћ* + С+ Ч+ J, N, X- - 7. 3 3 3 3 3 3 3 E I Є* Щ* - - - З- . 3. 7. 7. х K f t? е> $.
Н4
F= ф
#f )y ю Й
ЂD сQ *d + цL ?1 Ј{ N
Њ[ X ;%$ KS$ v% g{' B( ( Нw) еG* / ~./ "70 Ъ4 ;\8 Юz9 D; Ўi; *4<