Objectives: To develop a brief measure of patients' evaluation of the quality of diabetes care and to study predictors of consumers' rating of the quality of diabetes care.
Design: A prospective design.
Subjects: 176 adults with type 1 (39%) or type 2 (61%) diabetes.
Main measures: Demographic variables, HbA1c, number of diabetes complications, satisfaction with diabetes care, diabetes related distress, and fear of hypoglycaemia were assessed by self-report. In addition, satisfaction with diabetes care and evaluations about quality of the care were measured at 16 month follow up. Statistical analysis comprised principal component analyses, Cronbach's alpha, t tests, Pearson's correlation, and linear regression analyses.
Results: Results in the literature were used to develop the 14 items of the Patients' Evaluation of the Quality of Diabetes Care (PEQD) scale, assessing the most important aspects of the quality of diabetes care as delivered by the specialist in internal medicine (internist) and the diabetes nurse specialist (DNS). Two principal components analyses (internist/DNS) both yielded one 14 item factor with a high internal consistency. Satisfaction with diabetes care, fewer diabetes related complications, fewer treatment related problems, and a low level of worries about hypoglycaemia were predictors of a more positive evaluation of diabetes care delivered by the internist. Sociodemographic variables were not related to the patients' evaluations of the quality of diabetes care.
Conclusions: The PEQD comprises different aspects of quality of diabetes care and can be regarded as a suitable instrument for evaluating patients' judgements about the quality of their care.
Statistics from Altmetric.com
It has been shown in different patient groups that dissatisfaction with medical care is associated with non-compliance of the treatment regimen and discontinuation of care.1,2 Consumer satisfaction and the perceptions of patients with regard to the quality of their medical care can therefore be considered as important outcomes of health care and as essential elements of quality assessment and improvement.1,3
In patients with diabetes, discontinuation of care was found to be associated with worse glycaemic control4 and an increased risk of complications.5,6 In the new guidelines for quality diabetes care the European Diabetes Policy Group recently recommended provision of a system of quality development using feedback from diabetic patients on service performance with regular review.7
One method of obtaining patients' views on the quality of diabetes care is by the use of a validated self-report questionnaire. With such a standardised questionnaire, aspects of care that do not meet the expectations of patients can be detected by clinicians for internal quality improvement. This instrument would also be useful for researchers assessing patients' evaluations of the quality of their diabetes care.
Patients' evaluation of the quality of their diabetes care was planned to be one of the primary outcomes in a recent randomised controlled trial in which we tested the hypothesis that monitoring and discussion of psychological wellbeing by diabetes nurse specialists (DNSs) leads to improved outcomes of diabetes care.8–11 We searched Medline (1966–96) and PsycLit (1970–96) for an instrument that could be used in the trial to measure patients' evaluation of the quality of diabetes care using the words “consumer satisfaction”, “treatment satisfaction”, “diabetes”, and “psychometrics”. The literature search yielded two measures.12,13 The first12 was used to rate the performance of four professionals (hospital doctor, practice nurse, general practitioner, or diabetes liaison sister) in relation to seven aspects of diabetes care covering knowledge, communication, convenience, and accessibility. However, the development of the items of the instrument was not fully described. Moreover, a psychometric evaluation of the validity and reliability of this instrument was not provided. In the second study13 the literature and in depth interviews with 16 patients with diabetes and four diabetologists were used to develop a list of nine items associated with quality of diabetes care. Using this instrument, diabetes patients ranked statements about quality aspects of diabetes care to obtain a priority list. However, this instrument assesses the patients' evaluations of diabetes care priorities, and cannot be used to quantify the patients' rating of the quality of diabetes care.
We concluded that there was no brief validated instrument available to assess patients' evaluations of the quality of their diabetes care and therefore aimed to develop and validate a new questionnaire, the Patient's Evaluation of the Quality of Diabetes Care (PEQD). In this study we investigate the factor structure as well as the internal consistency and the convergent and discriminant validity of this new measure.
Development of the PEQD
Perhaps the greatest challenge facing us in the assessment of patient experiences with health care is focusing assessments on the most relevant unit of analysis.14 Some services (e.g. parking, cleanliness) are prominent in satisfaction questionnaires, but are seen as distinct from quality of care by most patients. Other issues such as “being involved in treatment decisions” and “being treated with respect” are fundamental issues for patients, yet these topics are frequently not included in satisfaction measures.14 Furthermore, studies suggest that the very concept of “satisfaction” is not adequate. Satisfaction only implies that expectations have been met. Patients can be satisfied with low quality care or can be dissatisfied with high quality care.14 Thus, in the construction of the PEQD, we have built upon the results of two studies that have investigated the health care priorities for diabetes.12,13 The following aspects of diabetes care were found to be valued the most in a sample of patients with type 2 diabetes (not on insulin): “getting clear information”, “health care provider knowledgeable about your problems with diabetes” and “easy to get hold of”, “easy to talk to”, “knowledgeable about diabetes”, “good at listening”.12 The second study reported that patients with type 1 or type 2 diabetes rated “treatment has favourable effect on diabetes” as the most important aspect of diabetes care, followed by “seeing same health care provider”, “getting all information needed”, “knowledgeable”, “enough time for the patient”, “provider tries to understand patient”, “patient can visit doctor soon after making appointment”, “patient can make choices between different types of treatment”.13 We used the results of both studies and a meta-analysis of the treatment satisfaction literature in general15 to develop 14 items (see Appendix). Subjects were requested to evaluate the quality of diabetes care provided by their doctor as well as their DNS during the past 12 months using two separate 14 item scales. A 5 point evaluation scale (poor to excellent) was used.16
A group of 176 patients with diabetes who had participated in a larger study in 199710,11 were invited to complete an additional set of questions in 1998. All subjects were patient members of the Dutch Diabetes Association (DVN, Diabetes Vereniging Nederland) and formed a heterogeneous sample who received medical treatment from different healthcare providers across the Netherlands. The subjects were matched to the experimental group of the aforementioned randomised clinical trial on sex, age, type of diabetes, and income.8 The study protocol was approved by the ethics committee of the Vrije Universiteit Medical Center.
At baseline (1997) in the earlier study10,11 subjects had already completed questions regarding demographic data, medication, glycaemic control, age at onset of diabetes, and complications of diabetes. Satisfaction with diabetes care was assessed at both interviews using the item: “How satisfied are you in general with your diabetes care?” This question could be responded to on a 5 point Likert scale ranging from “very satisfied” to “very dissatisfied”. Subjects also completed the Problem Areas in Diabetes (PAID) survey, a 20 item measure of diabetes related emotional distress.17,18 The subscales “diabetes related emotional problems” (12 items), “treatment related problems” (three items), “food related problems (three items), and “social support related problems” (two items) were also calculated.18 The 13 item worry scale of the Hypoglycaemia Fear Survey (HFS) was used to measure fear of hypoglycaemia.19,20 The subsales Negative Wellbeing, Energy, and Positive Wellbeing of Bradley's W-BQ12 were used to measure general psychological wellbeing.9–11 At follow up in 1998 the subjects filled out the PEQD scales for the internist (PEQD-I) and DNS (PEQD-DNS).
In case of 1–3 missing values (7–21% of all 14 items) for the PEQD, the subject's mean score on the remaining PEQD items in the same a priori scale was used to estimate the missing value(s). The paired Student's t test was used to compare the means of the 14 PEQD item scores regarding the care by the internist and the care provided by the DNS and to explore associations with demographic variables. Principal component analysis was used to investigate the structure of both scales. The Kaiser-Guttman criterion (eigenvalue g1) was used to decide on the number of factors to be retained.21 Homogeneity of these factors was determined by calculating item total correlations and internal consistency by Cronbach's alpha. Evidence of construct validity was sought by calculating Pearson's correlation coefficients between PEQD and overall satisfaction with diabetes care, age, and years of education. In case of skewed distribution, Spearman rank correlation was used. Stepwise linear regression analyses (with forward selection) were conducted to explore the relationship between baseline satisfaction with diabetes care, physical and psychological wellbeing, and the patients' evaluations of the quality of diabetes care at follow up.
All 176 subjects had already completed the baseline assessment and 155 (88%) of them returned the second follow up questionnaire a mean (SD) of 15.6 (0.7) months after the baseline assessment. The mean (SD) age was 55 (14) years; most of the patients were women and most had type 2 diabetes (treated with insulin, see table 1). One hundred and twenty four subjects completed all 14 questions regarding the treatment by their internist; 18 subjects had 1–3 missing values (12%) and 13 had more than three missing values. For the DNS items, 84 subjects had no missing values, 13 had 1–3 missing values (12%), and 10 had more than three missing values. Forty eight subjects did not complete the items evaluating the DNS because they did not visit a DNS during the previous 12 months.
Subjects who had visited the DNS during the 12 months preceding the follow up assessment were significantly younger than those who did not visit the DNS during this period (50 (15) years versus 59 (14) years, p<0.001). t tests and χ2 tests did not yield significant differences between the two groups in terms of sex, type of diabetes, number of complications, HbA1c, years of education, or satisfaction with diabetes care at follow up.
Principal components analyses
Principal components analyses of the 14 items of both scales yielded eigenvalues of 9.1 (PEQD-I) and 8.7 (PEQD-DNS), respectively. Other eigenvalues were h1.0, suggesting a one factor solution for both scales. Both one factor solutions explained 65.5% and 61.9% of the variance, respectively. The factor loadings ranged from 0.34 to 0.90 for the items regarding the internist's care and from 0.52 to 0.88 for the items evaluating the care of the DNS (table 2).
Patients' evaluations of diabetes care
Paired t tests indicated that respondents had a significantly more positive evaluation of the care delivered by the DNS than that delivered by the internist with regard to the following topics: waiting times, duration of consultations, clarity of information, amount and usefulness of information, opportunity to ask questions, and emotional support (table 2).
Homogeneity, internal consistency, and distribution of both scales
For the PEQD-I, Cronbach's alpha was 0.96 and the mean item total correlation was 0.77 (range 0.31–0.88); for the PEQD-DNS, Cronbach's alpha was 0.95 and the mean item total correlation was 0.74 (range 0.48–0.85). The 14 items were summed for both scales. Total scale scores were then transformed to a 0–100 scale using the formula: [(actual raw score – 14)/56] × 100.
The mean total score was significantly lower for the PEQD-I than for the PEQD-DNS (54.8 (20.7) versus 59.9 (18.8), p<0.001). PEQD-I scores ranged from 14.3 to 100 while PEQD-DNS scores ranged from 21.4 to 100, with skewnesses of 0.43 and 0.33, respectively. Pearson's correlation between PEQD-I and PEQD-DNS was 0.61 (p<0.001).
Associations with demographic characteristics
No significant differences (p<0.01, t test) in mean scores were seen in the PEQD items (internist or DNS) for the following variables: sex, type of diabetes, type of treatment for type 2 diabetes (diet/tablets versus insulin), marital status (having a partner versus single), or employment status (employed versus unemployed). Likewise, neither PEQD scale had a significant correlation with age or years of education.
Associations with self-reported psychological and biomedical variables
The PEQD-I scale was significantly associated with the 20 item overall scale of the PAID survey (–0.24, p=0.006) and its four subscales: diabetes related emotional problems (–0.21, p=0.017), treatment related problems (–0.37, p=0.000), food related problems (–0.18, p=0.041), and social support related problems (–0.18, p=0.037) and –0.33 (p<0.001) with the HFS worry scale. The correlations between the PEQD-DNS scale and the PAID (sub)scales and the HFS worry scale did not exceed 0.15 and did not reach statistical significance.
Both PEQD scales had a moderate correlation with overall satisfaction with diabetes care at follow up (internist 0.54, DNS 0.44, p<0.001). Table 3 shows that baseline satisfaction with diabetes care was positively associated with the following variables at follow up: PEQD-I (β=0.23), PEQD-DNS (β=0.31), and satisfaction with diabetes care at follow up (β=0.43), explaining 11%, 8%, and 22% of the variance, respectively. The number of diabetes related complications was significantly associated with PEQD-I and satisfaction with care at follow up. With PEQD-DNS as the dependent variable, the number of diabetes related complications was comparable to the other models (β=–0.22) but was not significant (p=0.09).
Patients who reported a higher level of treatment related problems and/or worries about hypoglycaemic episodes at baseline evaluated the quality of care delivered by their internist more negatively at follow up. However, neither variable was significantly related to PEQD-DNS or satisfaction with diabetes care at follow up. A higher level of diabetes related emotional problems at baseline was positively related with a more favourable evaluation of the care delivered by the DNS. The patients' evaluation of the quality of diabetes care was not significantly associated with self-reported HbA1c.
The outcome of this research is a brief questionnaire assessing the patients' judgement about the quality of his or her diabetes care. This instrument appeared to have a clear structure. Analyses yielded a one factor solution with adequate reliability, as indicated by high internal consistencies and high item total correlations. In the development of the PEQD scale we tried to maximise face and content validity by retaining those issues about quality of diabetes care that patients and diabetologists have identified as most important.12,13,22 Because we aimed to develop a brief instrument, some aspects of quality of care that were thought to be less relevant were not included—for example, facilities, hygiene, prevention of superfluous care. A possible disadvantage of this approach is that some of the topics that were not included may be highly relevant for other samples with a different educational, racial, or ethnic background compared with the samples of the two previous qualitative studies. Most patients with type 1 diabetes are treated in outpatient clinics, while most of those with type 2 diabetes depend on general practice for routine diabetes care. As the aspects of care that were valued the most by diabetic patients were found to be comparable in general practice and in the outpatient setting,12,13 we believe that the PEQD can be used to evaluate the care delivered by the internist, the diabetes nurse specialist, and also by the general practitioner. To do so, the word “internist” can simply be replaced by the words “diabetes nurse specialist” or “general practitioner”. However, further research is needed to investigate the face and content validity of the PEQD in different patient groups and healthcare settings.
There are no brief validated instruments available to assess patients' evaluations of the quality of diabetes care.
A new instrument, the PEQD (Patient's Evaluations of the Quality of Diabetes Care), was developed using results in the literature regarding healthcare priorities of diabetic patients and diabetologists.
The results of the present study support the validity and reliability of the PEQD.
Further research is needed to evaluate the PEQD in different patient groups and healthcare settings.
The distributions of almost all items of both versions of the PEQD were close to normal and both overall scales proved to have very little skew. As a consequence, it is likely that the PEQD is sufficiently sensitive to detect positive and negative changes in patients' evaluations of their quality of diabetes care. However, the test-retest reliability and sensitivity to change of this new instrument still has to be determined.
In this study we found no significant correlations between demographic characteristics and patients' perceptions of the quality of diabetes care. This result is in line with the findings of Hall and Dornan23 who concluded that sociodemographic characteristics are a minor predictor of satisfaction with care, a concept that is closely related to the patients' evaluation of the quality of care.
The number of diabetes related complications and the level of diabetes related distress showed weak associations with the patients' evaluations of the quality of their diabetes care. This finding partly corroborates the finding that poor overall health in chronically ill patients was a minor predictor of a less positive judgement of the quality of general practice care.24 The association between the number of complications and a more negative evaluation of diabetes care can be explained in many ways. For example, diabetes related complications may in fact be a consequence of low quality medical care. Conversely, as competent medical treatment cannot avoid the development of diabetes related complications, sicker patients may also be inclined to blame the internist and give a more negatively biased evaluation of the diabetes care. Another plausible explanation is that healthcare providers react to sicker patients in ways that produce a more negative evaluation of the quality of their care—for example, in patients with depression, patients who are excessive smokers, or those who are unwilling to give up an unhealthy diet. Further research is needed to test these hypotheses.
The finding that the diabetes nurse specialists had more favourable evaluations regarding duration of consultations, amount and usefulness of information, opportunity to ask questions, and emotional support than the internists provides further support for the validity of the PEQD because the nurses generally spend more time with patients and one of their primary tasks is to provide information to the patients.
In conclusion, the results of this study strongly support the validity and reliability of the PEQD. Although further research is needed to investigate other psychometric characteristics of the PEQD, our findings suggest that clinicians and researchers can now use this instrument to assess patients' evaluations of the quality of their diabetes care in a valid and reliable way.
The authors acknowledge the members and the board of the Dutch Diabetes Association (Diabetes Vereniging Nederland) for their cooperation in this study. The study was financially supported by the Dutch Diabetes Research Foundation (Diabetes Fonds Nederland, grant 95.805).
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.