Comparison of health care professionals’ self-assessments of standards of care and patients’ opinions on the care they received in hospital: observational study
- 1Department of Public Health and Medical Informatics, Faculté de Médecine Broussais Hôtel Dieu and Hôpital Européen Georges Pompidou, Paris, France
- 2Centre d’Investigations Cliniques 9201, Assistance-Publique des hôpitaux de Paris/INSERM, Hôpital Européen Georges Pompidou, Paris, France
- 3Public Health Unit, Hôpital Paul Brousse, Villejuif, France
- 4Department of Biostatistics and Medical Informatics, Hôpital Cochin, Paris, France
- Correspondence to: Dr D P Durieux Santé Publique et Informatique Médicale, Faculté de Médecine Broussais Hôtel Dieu, 15 rue de l’Ecole de Médecine, 75006 Paris, France;
- Accepted 7 March 2004
Objectives: To compare the views of healthcare professionals and patients regarding compliance with standards of care concerning patient information.
Design: Self-rated questionnaire survey.
Setting: Nine wards in short stay French hospitals.
Participants: 939 patients and 359 healthcare professionals (physicians, nurses, assistants and other professionals).
Main outcome measure: Patients’ and healthcare professionals’ views of compliance with 20 standards of patient care described in the French accreditation manual. Comparison of the rank order of the standards within the two samples.
Results: The response rate was 61.5% in the patient group and 85.8% in the healthcare professionals. The rank orders for the 20 items were similar in both groups (Spearman rank order correlation 0.6, p = 0.004). The two items ranked highest by healthcare professionals (“consent request for a surgical procedure” and “the doctors ask the visitors to leave the room before examining a patient”) were also the two ranked highest by the patients. Three items were ranked low by both groups: “consent request for students to be present”, “health education given to patients”, and “possibility to express satisfaction during discharge”. Patients were more satisfied with their pain management than were healthcare providers. Professionals were more satisfied with the social services than the patients.
Conclusion: There are both similarities and differences between patients’ and healthcare professionals’ views of care. Accurate assessments of quality performed during the accreditation procedure require that both patients’ and professionals’ views be taken into account.
Healthcare professionals’ assessments of the care provided to patients and patients’ opinions about the care they receive represent two approaches which have been proposed to measure quality in the hospital setting.1 Patient satisfaction surveys provide valuable information for those trying to improve quality.2–4 Self-assessment of quality of care by healthcare professionals has been included in the accreditation procedure, a form of external peer review of organisational processes and structures adopted by several countries including France.5–8 Professional self-assessment is a fundamental step in the accreditation procedure in which the professionals compare their own practices with prescribed standards of care.9 Self-assessment tells whether those standards are being achieved and helps the identification of practices that need improvement.
It has been shown that the quality of information provided to patients is a major determinant of their satisfaction.4 However, it has been suggested that healthcare professionals have limited awareness of patients’ opinions regarding the provision and receipt of health related information.10,11 In this study we considered the following question: “How do healthcare professionals’ assessments of standards of care concerning patient information compare with patients’ opinions?”
Study site and population
The study was conducted in 2001 among healthcare professionals and patients in nine volunteer wards of five short stay hospitals in the Paris region (three public teaching hospitals, one public general hospital, and one private hospital). In the French healthcare system, two thirds of hospitals are publicly owned and one third is private. Patients are insured by different mandatory health insurance funds and have access to hospitals, whether public or private. The medical specialities of the wards selected for study were haematology (n = 1), chest diseases (n = 2), internal medicine (n = 1), gynaecology (n = 2), cardiology (n = 1), hypertension (n = 1), and general surgery (n = 1).
In 1996 a mandatory accreditation procedure was introduced in France covering all public and private healthcare organisations. Inspired by models from English speaking countries, its objective is to ensure that safety conditions and quality of patient care are adequately addressed.6–9 A French national agency, Agence nationale d’accréditation et d’évaluation en santé (ANAES), responsible for this procedure, has described the accreditation process in an accreditation manual which contains a total of 300 standards against which continuing progress in quality will be evaluated in each healthcare organisation.12 An accreditation standard is defined as the statement of an expectation or requirement which makes it possible to deliver quality care or services. The standards were drafted by working groups (including healthcare professionals and patient representatives), then tested in a sample of healthcare organisations and reviewed by numerous French groups of healthcare professionals and patient organisations.8
Twenty five standards dealing with patient information were selected from the accreditation manual. From this list and from the items of a satisfaction questionnaire validated in a previous study13 we selected 20 items assessing various aspects of care: information concerning the health status and care of the patient (items 1–8); physical/moral well being of the patient (items 9–12); relationship with the staff (items 13–16), and information concerning the discharge of the patient (items 17–20).
Based on these items, two self-administered questionnaires were designed, one for the patients and the other for healthcare professionals. The two questionnaires comprised the same questions but were worded differently. Patients were asked to evaluate the care received during their stay in hospital. Healthcare professionals were asked about the care provided in their wards, in a manner similar to that used during the accreditation procedure. For example, the item written “I was informed in advance about the medical procedures that were planned” for patients was expressed as “patients are informed in advance about the medical procedures that are planned” for healthcare professionals.
Patients were asked to evaluate their satisfaction with care on a 5-point scale (1 = absolutely, 2 = quite, 3 = not really, 4 = not at all, and 5 = not concerned). Healthcare professionals were asked to give their opinion on a different 5-point scale equivalent to the French accreditation scale because they were asked to consider patients treated in the ward as a whole (1 = always or very often (meaning “we satisfy the standard)”, 2 = quite often (“we largely satisfy the standard”), 3 = sometimes (“we partially satisfy the standard”), 4 = very rarely or never (“we do not satisfy the standard”), and 5 = not concerned).
Patient characteristics (age, sex, educational level, perceived health status, duration of the principal disease, type of hospital care: admission, type of service) and health professional characteristics (age, sex, time spent in the study hospital, time spent in the study ward, type of activity) were included in the questionnaires.
The patient questionnaire was sent by post 7–10 days after discharge to 100 consecutive patients from each department, except for the private hospital (n = 200). those who were transferred to another ward following their stay in a study ward were excluded from the study. Patients were asked to complete the questionnaire at home and to return it by post using a prepaid envelope. When patients did not return their questionnaire two reminders were sent 10 days apart.
In each participating ward, independently of the patient survey, all healthcare professionals (physicians, chief nurses, nurses, care assistants, physiotherapists, secretaries, midwives, etc) were asked to complete the healthcare professionals’ questionnaire.
According to French policy, this study was exempt from medical ethics committee approval. In addition, questionnaires were built and delivered in order to collect no personal information concerning patients or healthcare professionals (such as age, name, or date of birth).
Analysis of data
For each question participants who did not answer the question or answered “not concerned” were excluded. Each question was scored on a 5-point scale (1 for “absolutely” or “always”, 2 for “quite” or “quite often” and so on). For each question we computed the mean scores in the patient and the healthcare professional samples. Higher values indicate a lower quality.
We also calculated the frequency of each response and computed, for each question, the mean percentage of answers “absolutely” (patients) and the mean percentage of answers “always or very often” (healthcare professionals).
Mean scores obtained for each question were ranked from the best to the worst score within each sample. The Spearman’s rank correlation test was used to study the relationship between patients’ and professionals’ responses: the rank order of the 20 questions in the patient sample was compared with the rank order in the healthcare professional sample and in each category of healthcare professionals (physicians, nurses, care assistants).
Of the 939 patients and 359 healthcare professionals who received a questionnaire, 577 patients (participation rate 61.5%) and 308 healthcare professionals (participation rate 85.8%) completed and returned it. A total of 58 physicians, 149 nurses, 62 care assistants, and 39 other professionals were included in the study. Tables 1 and 2 present the characteristics of the two samples.
Comparisons of views of healthcare professionals and patients
Comparisons of assessments made by healthcare professionals and patients are given in table 3. The correlation between patients’ and professionals’ evaluations was 0.6 (p = 0.004, Spearman test)—that is, the rank orders for the 20 items were similar. When groups of health professionals were considered individually, the value of the Spearman test was 0.76 (p = 0.001), 0.47 (p = 0.038), and 0.47 (p = 0.04) for physicians, nurses and care assistants, respectively.
The two items ranked highest by healthcare professionals were also the two ranked highest by the patients—namely, “consent request for a surgical procedure” and “the doctors ask the visitors to leave the room before examining a patient”. Item 13 in table 3 (“identity of the members of the staff known by patients”) was ranked third by patients and fourth by healthcare professionals. Three items were ranked low by both groups: “consent request for students to be present” (ranked 18 by patients and 20 by healthcare professionals), “health education given to patients” (ranked 15 by patients and 19 by healthcare professionals), and “possibility to express satisfaction during discharge” (ranked 16 by both groups).
We also observed some large differences in rank order of evaluations. The largest differences were observed for two items: “patients are asked about pain during hospitalisation” (score of 1.60, 72.7% “absolutely” and ranked 9 by patients compared with a score of 2.13, 29.8% “always”, and ranked 18 by healthcare professionals) and “patients were asked if they needed social assistance” (ranked 20 by patients and 12 by healthcare professionals). More generally, patients evaluated more positively questions related to the care they received (questions 1–6) and more negatively questions related to discharge (questions 17–20) than healthcare professionals.
This study has shown both similarities and differences between the assessments of patients and healthcare professionals of standards of care that concern the provision of information to patients. Overall, the rank order of the standards’ evaluation was similar in the two populations analysed. Three items (related to consent given when requested for care and relation with the staff) were evaluated positively and three items (related to consent requested for students to be present, health education, and evaluation of patient satisfaction) were evaluated negatively by both groups. Patients’ and healthcare professionals’ evaluations differed for some items (pain management, for which patients were more satisfied with compliance with standards than healthcare professionals, and social assistance for which the reverse pattern was observed).
Reasons for measuring the quality of care include obtaining more detailed information about patient care and identifying potential areas for improvement. Our study has shown that healthcare professionals alone cannot identify all quality problems related to the provision of patient information. Caregivers have a tendency to underestimate the quality of non-technical aspects of care and to give priority to technical aspects of care.14–16 It has also been shown, as in our study, that psychosocial issues are often left unaddressed.17
We were surprised by the very large discrepancy between patients’ and healthcare professionals’ scores concerning the question on pain assessment. A previous study performed in Sweden had shown that general satisfaction with postoperative pain management was rated higher by patients than by nurses.18 However, several French studies have concluded that pain management was inadequate.19–22 In 1998 the improvement in pain management in hospitals was considered to be a priority of the Ministry of Health and a national programme was proposed together with a campaign using mass media to increase public awareness. It is therefore likely that a certain amount of culpability persists among healthcare professionals who believe that they must continue to improve pain management in their patients. However, it is also possible that patients underestimate their pain, believing that it is a natural part of their disease and remaining uncomplaining. In our study only 8% of patients considered that their health status would worsen (table 1).
Previous studies of patient satisfaction have shown that patients are reluctant to express dissatisfaction.23 Precautions were therefore taken to avoid biased responses: anonymous questionnaires were sent home after release from the hospital to demonstrate a lack of involvement of any of their care providers.24–27
Improvements in patients’ health after hospitalisation could also explain their positive view.28–30 Our study was focused on patient information. Previous studies have shown that lack of information was a major cause of patient dissatisfaction4,10 and that patients value the effective communication of health related information much more than physicians.10 There are also discrepancies between what physicians report having communicated to patients and what patients remember being told.31
The overall rank order between patients’ and healthcare professionals’ evaluation of quality of care based on the compliance to 20 standards of care related to patient information was similar.
Patients and professionals agreed on what is best (consent was requested before surgery, visitors were asked to leave the room before an examination) and worst (consent requested for students to be present, health education, and evaluation of patient satisfaction).
Patients were more satisfied than professionals with pain management.
Patients felt that more attention should be paid to non-technical aspects of care, to information on what can be done after discharge, and to help with psychosocial problems.
Our study has some limitations. Firstly, although the patient response rate (61.5%) was above average for postal surveys,32 we have no data concerning the non-respondents who may have had very different views on the information they received from health professionals. Secondly, the data obtained from healthcare professionals did not refer to the care of specific patients but to the usual practices of their wards. This lack of patient specific information makes it difficult to establish a direct link between the responses of patients and healthcare professionals. Thirdly, the mix of health professionals in our study would appear to be a limitation. However, the accreditation standards require that all staff of the hospital (“from the doctors to the secretaries”) must be knowledgeable about these standards. In our study similar response patterns were observed in each category of healthcare professional when compared separately with patients. Fourthly, these results reflect a comparison of healthcare professionals’ and patients’ assessments of selected aspects of patient information provided in a specific healthcare system. It is not known whether they can be generalised to other topics or other systems.
While accreditation systems provide standards of care for the guidance of healthcare professionals, the validity of their self-assessment of quality should be questioned.33,34 We know that there is no “magic bullet” to improve the quality of hospital care and that, as has already been shown in general practice,35 both patient and healthcare professional evaluations must be considered together if a high quality of care is to be achieved.
The authors thank all the healthcare professionals and patients who agreed to participate in the study, particularly the chief nurses and the heads of the participating wards.
Funding: The study was supported by the Direction des Hôpitaux, Ministère de l’Emploi et de la Solidarité (Projets Hospitaliers de Recherche Clinique, Délégation à la Recherche Clinique d’Ile de France, grant no AOM 98–111).
Conflict of interest: none
Authorship: PD had the original idea, designed the study and drafted the manuscript. AB and JC performed the statistical analysis. All authors analysed and interpreted the data, revised the manuscript critically for content and approved the final version.