METHODS

Development of the PEQD

Perhaps the greatest challenge facing us in the assessment of patient experiences with health care is focusing assessments on the most relevant unit of analysis.¹⁴ Some services (e.g. parking, cleanliness) are prominent in satisfaction questionnaires, but are seen as distinct from quality of care by most patients. Other issues such as “being involved in treatment decisions” and “being treated with respect” are fundamental issues for patients, yet these topics are frequently not included in satisfaction measures.¹⁴ Furthermore, studies suggest that the very concept of “satisfaction” is not adequate. Satisfaction only implies that expectations have been met. Patients can be satisfied with low quality care or can be dissatisfied with high quality care.¹⁴ Thus, in the construction of the PEQD, we have built upon the results of two studies that have investigated the health care priorities for diabetes.^12,13 The following aspects of diabetes care were found to be valued the most in a sample of patients with type 2 diabetes (not on insulin): “getting clear information”, “health care provider knowledgeable about your problems with diabetes” and “easy to get hold of”, “easy to talk to”, “knowledgeable about diabetes”, “good at listening”.¹² The second study reported that patients with type 1 or type 2 diabetes rated “treatment has favourable effect on diabetes” as the most important aspect of diabetes care, followed by “seeing same health care provider”, “getting all information needed”, “knowledgeable”, “enough time for the patient”, “provider tries to understand patient”, “patient can visit doctor soon after making appointment”, “patient can make choices between different types of treatment”.¹³ We used the results of both studies and a meta-analysis of the treatment satisfaction literature in general¹⁵ to develop 14 items (see Appendix). Subjects were requested to evaluate the quality of diabetes care provided by their doctor as well as their DNS during the past 12 months using two separate 14 item scales. A 5 point evaluation scale (poor to excellent) was used.¹⁶

Subjects

A group of 176 patients with diabetes who had participated in a larger study in 1997^10,11 were invited to complete an additional set of questions in 1998. All subjects were patient members of the Dutch Diabetes Association (DVN, Diabetes Vereniging Nederland) and formed a heterogeneous sample who received medical treatment from different healthcare providers across the Netherlands. The subjects were matched to the experimental group of the aforementioned randomised clinical trial on sex, age, type of diabetes, and income.⁸ The study protocol was approved by the ethics committee of the Vrije Universiteit Medical Center.

Other measures

At baseline (1997) in the earlier study^10,11 subjects had already completed questions regarding demographic data, medication, glycaemic control, age at onset of diabetes, and complications of diabetes. Satisfaction with diabetes care was assessed at both interviews using the item: “How satisfied are you in general with your diabetes care?” This question could be responded to on a 5 point Likert scale ranging from “very satisfied” to “very dissatisfied”. Subjects also completed the Problem Areas in Diabetes (PAID) survey, a 20 item measure of diabetes related emotional distress.^17,18 The subscales “diabetes related emotional problems” (12 items), “treatment related problems” (three items), “food related problems (three items), and “social support related problems” (two items) were also calculated.¹⁸ The 13 item worry scale of the Hypoglycaemia Fear Survey (HFS) was used to measure fear of hypoglycaemia.^19,20 The subsales Negative Wellbeing, Energy, and Positive Wellbeing of Bradley's W-BQ12 were used to measure general psychological wellbeing.^9–11 At follow up in 1998 the subjects filled out the PEQD scales for the internist (PEQD-I) and DNS (PEQD-DNS).

Statistical analyses

In case of 1–3 missing values (7–21% of all 14 items) for the PEQD, the subject's mean score on the remaining PEQD items in the same a priori scale was used to estimate the missing value(s). The paired Student's t test was used to compare the means of the 14 PEQD item scores regarding the care by the internist and the care provided by the DNS and to explore associations with demographic variables. Principal component analysis was used to investigate the structure of both scales. The Kaiser-Guttman criterion (eigenvalue g1) was used to decide on the number of factors to be retained.²¹ Homogeneity of these factors was determined by calculating item total correlations and internal consistency by Cronbach's alpha. Evidence of construct validity was sought by calculating Pearson's correlation coefficients between PEQD and overall satisfaction with diabetes care, age, and years of education. In case of skewed distribution, Spearman rank correlation was used. Stepwise linear regression analyses (with forward selection) were conducted to explore the relationship between baseline satisfaction with diabetes care, physical and psychological wellbeing, and the patients' evaluations of the quality of diabetes care at follow up.

RESULTS

All 176 subjects had already completed the baseline assessment and 155 (88%) of them returned the second follow up questionnaire a mean (SD) of 15.6 (0.7) months after the baseline assessment. The mean (SD) age was 55 (14) years; most of the patients were women and most had type 2 diabetes (treated with insulin, see table 1). One hundred and twenty four subjects completed all 14 questions regarding the treatment by their internist; 18 subjects had 1–3 missing values (12%) and 13 had more than three missing values. For the DNS items, 84 subjects had no missing values, 13 had 1–3 missing values (12%), and 10 had more than three missing values. Forty eight subjects did not complete the items evaluating the DNS because they did not visit a DNS during the previous 12 months.

Subjects who had visited the DNS during the 12 months preceding the follow up assessment were significantly younger than those who did not visit the DNS during this period (50 (15) years versus 59 (14) years, p<0.001). t tests and χ² tests did not yield significant differences between the two groups in terms of sex, type of diabetes, number of complications, HbA_1c, years of education, or satisfaction with diabetes care at follow up.

Principal components analyses

Principal components analyses of the 14 items of both scales yielded eigenvalues of 9.1 (PEQD-I) and 8.7 (PEQD-DNS), respectively. Other eigenvalues were h1.0, suggesting a one factor solution for both scales. Both one factor solutions explained 65.5% and 61.9% of the variance, respectively. The factor loadings ranged from 0.34 to 0.90 for the items regarding the internist's care and from 0.52 to 0.88 for the items evaluating the care of the DNS (table 2).

Patients' evaluations of diabetes care

Paired t tests indicated that respondents had a significantly more positive evaluation of the care delivered by the DNS than that delivered by the internist with regard to the following topics: waiting times, duration of consultations, clarity of information, amount and usefulness of information, opportunity to ask questions, and emotional support (table 2).

Homogeneity, internal consistency, and distribution of both scales

For the PEQD-I, Cronbach's alpha was 0.96 and the mean item total correlation was 0.77 (range 0.31–0.88); for the PEQD-DNS, Cronbach's alpha was 0.95 and the mean item total correlation was 0.74 (range 0.48–0.85). The 14 items were summed for both scales. Total scale scores were then transformed to a 0–100 scale using the formula: [(actual raw score – 14)/56] × 100.

The mean total score was significantly lower for the PEQD-I than for the PEQD-DNS (54.8 (20.7) versus 59.9 (18.8), p<0.001). PEQD-I scores ranged from 14.3 to 100 while PEQD-DNS scores ranged from 21.4 to 100, with skewnesses of 0.43 and 0.33, respectively. Pearson's correlation between PEQD-I and PEQD-DNS was 0.61 (p<0.001).

Associations with demographic characteristics

No significant differences (p<0.01, t test) in mean scores were seen in the PEQD items (internist or DNS) for the following variables: sex, type of diabetes, type of treatment for type 2 diabetes (diet/tablets versus insulin), marital status (having a partner versus single), or employment status (employed versus unemployed). Likewise, neither PEQD scale had a significant correlation with age or years of education.

Associations with self-reported psychological and biomedical variables

The PEQD-I scale was significantly associated with the 20 item overall scale of the PAID survey (–0.24, p=0.006) and its four subscales: diabetes related emotional problems (–0.21, p=0.017), treatment related problems (–0.37, p=0.000), food related problems (–0.18, p=0.041), and social support related problems (–0.18, p=0.037) and –0.33 (p<0.001) with the HFS worry scale. The correlations between the PEQD-DNS scale and the PAID (sub)scales and the HFS worry scale did not exceed 0.15 and did not reach statistical significance.

Both PEQD scales had a moderate correlation with overall satisfaction with diabetes care at follow up (internist 0.54, DNS 0.44, p<0.001). Table 3 shows that baseline satisfaction with diabetes care was positively associated with the following variables at follow up: PEQD-I (β=0.23), PEQD-DNS (β=0.31), and satisfaction with diabetes care at follow up (β=0.43), explaining 11%, 8%, and 22% of the variance, respectively. The number of diabetes related complications was significantly associated with PEQD-I and satisfaction with care at follow up. With PEQD-DNS as the dependent variable, the number of diabetes related complications was comparable to the other models (β=–0.22) but was not significant (p=0.09).

Patients who reported a higher level of treatment related problems and/or worries about hypoglycaemic episodes at baseline evaluated the quality of care delivered by their internist more negatively at follow up. However, neither variable was significantly related to PEQD-DNS or satisfaction with diabetes care at follow up. A higher level of diabetes related emotional problems at baseline was positively related with a more favourable evaluation of the care delivered by the DNS. The patients' evaluation of the quality of diabetes care was not significantly associated with self-reported HbA_1c.

DISCUSSION

The outcome of this research is a brief questionnaire assessing the patients' judgement about the quality of his or her diabetes care. This instrument appeared to have a clear structure. Analyses yielded a one factor solution with adequate reliability, as indicated by high internal consistencies and high item total correlations. In the development of the PEQD scale we tried to maximise face and content validity by retaining those issues about quality of diabetes care that patients and diabetologists have identified as most important.^12,13,22 Because we aimed to develop a brief instrument, some aspects of quality of care that were thought to be less relevant were not included—for example, facilities, hygiene, prevention of superfluous care. A possible disadvantage of this approach is that some of the topics that were not included may be highly relevant for other samples with a different educational, racial, or ethnic background compared with the samples of the two previous qualitative studies. Most patients with type 1 diabetes are treated in outpatient clinics, while most of those with type 2 diabetes depend on general practice for routine diabetes care. As the aspects of care that were valued the most by diabetic patients were found to be comparable in general practice and in the outpatient setting,^12,13 we believe that the PEQD can be used to evaluate the care delivered by the internist, the diabetes nurse specialist, and also by the general practitioner. To do so, the word “internist” can simply be replaced by the words “diabetes nurse specialist” or “general practitioner”. However, further research is needed to investigate the face and content validity of the PEQD in different patient groups and healthcare settings.

The distributions of almost all items of both versions of the PEQD were close to normal and both overall scales proved to have very little skew. As a consequence, it is likely that the PEQD is sufficiently sensitive to detect positive and negative changes in patients' evaluations of their quality of diabetes care. However, the test-retest reliability and sensitivity to change of this new instrument still has to be determined.

In this study we found no significant correlations between demographic characteristics and patients' perceptions of the quality of diabetes care. This result is in line with the findings of Hall and Dornan²³ who concluded that sociodemographic characteristics are a minor predictor of satisfaction with care, a concept that is closely related to the patients' evaluation of the quality of care.

The number of diabetes related complications and the level of diabetes related distress showed weak associations with the patients' evaluations of the quality of their diabetes care. This finding partly corroborates the finding that poor overall health in chronically ill patients was a minor predictor of a less positive judgement of the quality of general practice care.²⁴ The association between the number of complications and a more negative evaluation of diabetes care can be explained in many ways. For example, diabetes related complications may in fact be a consequence of low quality medical care. Conversely, as competent medical treatment cannot avoid the development of diabetes related complications, sicker patients may also be inclined to blame the internist and give a more negatively biased evaluation of the diabetes care. Another plausible explanation is that healthcare providers react to sicker patients in ways that produce a more negative evaluation of the quality of their care—for example, in patients with depression, patients who are excessive smokers, or those who are unwilling to give up an unhealthy diet. Further research is needed to test these hypotheses.

The finding that the diabetes nurse specialists had more favourable evaluations regarding duration of consultations, amount and usefulness of information, opportunity to ask questions, and emotional support than the internists provides further support for the validity of the PEQD because the nurses generally spend more time with patients and one of their primary tasks is to provide information to the patients.

In conclusion, the results of this study strongly support the validity and reliability of the PEQD. Although further research is needed to investigate other psychometric characteristics of the PEQD, our findings suggest that clinicians and researchers can now use this instrument to assess patients' evaluations of the quality of their diabetes care in a valid and reliable way.