Objective: To describe and compare patient and nurse assessments of the quality of care in postoperative pain management, to investigate differences between subgroups of patients, and to compare patient assessments in different departments.
Design: Patient and nurse questionnaires.
Setting: Five surgical wards in general surgery, orthopaedics, and gynaecology in a central county hospital in Sweden.
Sample: Two hundred and nine inpatients and 64 registered nurses. The response rates were 96% for the patients and 99% for the nurses; there were 196 paired patient-nurse assessments.
Method: The Strategic and Clinical Quality Indicators in Postoperative Pain Management patient questionnaire was used which comprises14 items in four subscales (communication, action, trust, and environment). The items were scored on a 5 point scale with higher values indicating a higher quality of care. Five complementary questions on levels of pain intensity and overall satisfaction with pain relief were scored on an 11 point scale. Twelve of the 14 items in the patient questionnaire and two of the complementary questions were adjusted for use in the nurse questionnaire.
Results: The patients’ mean (SD) score on the total scale (scale range 14–70) was 58.6 (8.9) and the nurses’ mean (SD) score (scale range 12–60) was 48.1 (6.2). The percentage of patients who scored 1 or 2 for an individual item (disagreement) ranged from 0.5% to 52.0%, while for nurses the percentage ranged from 0.0% to 34.8%. Forty two patients (24%) reported more pain than they expected; these patients assessed the quality of care lower. There were differences between patient and nurse assessments concerning the environment subscale, the question on overall satisfaction, and patients’ experience of worst possible pain intensity.
Conclusion: The results provided valuable baseline data and identified important areas for quality improvement in postoperative pain management.
- postoperative pain
- quality improvement
- patient assessment
- nurse assessment
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Reasons for measuring the quality of care include obtaining more detailed information about patient care, determining whether standards are being achieved, identifying potential areas for improvement, and informing purchasers and thereby securing resources for future services. Good quality of care is considered to be the right of all patients and the responsibility of all staff within the hospital. Important clinical aspects of care may vary from one department to another, based on the patient population and services provided. The aspects to be measured—the quality indicators—can be considered from four points of view: (1) high volume activities, (2) high risk activities, (3) problem prone activities, and (4) high cost activities.1 Postoperative pain management in a surgical department seems to meet these criteria.
Several authors have reported that the treatment of postoperative pain is inadequate for many patients, some of whom still suffer moderate to severe pain.2–,4 Klopfenstein et al5 considered the reasons for poor postoperative pain management as insufficient education and training for staff and patients, lack of communication between staff and patients, divergent attitudes, absence of systematic recordings, pain assessment only at rest, and lack of public awareness. Both nurses and physicians have a tendency to underestimate the patients’ pain and pain relief.5–,7 It is, however, also important to determine how nurses assess the quality of care in postoperative pain management in a broader perspective and to examine whether these assessments correspond with those of the patients. It is therefore important to identify individual patient’s opinions and experiences of the care provided and to compare this with nurses’ evaluations of their own work in a particular patient.
Do nurses have the same opinion as patients of the quality of care?
Do nurses assess their own work as not being sufficient to meet the standards for postoperative pain management?
Areas for improvement could be identified from both a patient perspective and from a caregiver perspective. Furthermore, to improve postoperative pain management it is important to evaluate possible differences in quality of care between subgroups of patients and in different clinical settings.
An instrument to measure the quality of postoperative pain management, Strategic and Clinical Quality Indicators in Postoperative Pain Management, has recently been developed8 from a qualitative approach using focus group interviews.9 The items which emerged from the interviews were tested for content validity10,11 using the method proposed by Fehring.12 The items were assessed as being (a) essential for the quality of care in postoperative pain management, (b) realistic to carry out, and (c) possible for nurses to influence management.10,11 To explore the underlying structure a factor analysis was performed from which four factors (subscales) emerged—communication, action, trust, and environment—with 14 items in total.8,13 A Cronbach’s coefficient alpha of 0.84 and an average inter-item correlation of 0.32 confirmed internal consistency reliability, and convergent validity suggested initial support for the instrument.
The present study used this instrument supplemented with questions on pain intensity levels and general satisfaction to:
describe and compare patient and nurse assessments of the quality of care in postoperative pain management;
investigate differences between subgroups of patients;
compare patient assessments in general surgery, orthopaedics, and gynaecology.
Data were collected from paired patient-nurse assessments using two questionnaires, one for patients and one for nurses.
Two hundred and nine inpatients from five orthopaedic, gynaecological, and general surgical wards in a central county hospital in Sweden were included in the study. The patients answered a questionnaire on their second postoperative day and inclusion criteria were: scheduled for an elective surgical procedure, oriented to person and place, able to understand Swedish, at least 18 years of age, and a minimum of 24 hours in the wards after the surgical procedure. Data were collected on two or three weekdays per week over 4 months from September to December 1999 and included all patients who met the inclusion criteria.
Sixty four registered nurses from the same five surgical wards as the patients were included in the study.
The patient questionnaire was the instrument Strategic and Clinical Quality Indicators which consisted of 14 items, two complementary questions concerning general patient satisfaction, and three complementary questions concerning pain intensity level (Appendix 1). The complementary questions concerning pain intensity levels were from the Brief Pain Inventory.14
The composition of the questionnaire was derived from “Measurements sets” presented by the Joint Commission in the National Library of Healthcare Indicators15 which have been used to assess the quality of care of, for example, diabetes care and breast cancer care and embraced (1) steps to good care, (2) satisfaction, and (3) results. The instrument Strategic and Clinical Quality Indicators in Postoperative Pain Management measures steps to good care and the complementary questions measure general satisfaction and results. Demographic variables collected from the patient records included age, sex, and whether or not the patient received epidural analgesia and were used to identify subgroups of patients.
The nurse questionnaire concerned the nurse’s opinion on how an individual patient’s care was performed. Twelve of the 14 statements in the instrument used in the patient questionnaire were used and adjusted in a nurse questionnaire. For example, “I received support or help to find a comfortable position in bed to help avoid pain” (patient questionnaire) was amended to “The patient received support or help in finding a comfortable position in bed to help avoid pain” (nurse questionnaire). Two items were excluded because it was difficult to amend them satisfactorily, and two additional questions were asked: “Worst pain the patients have experienced in the last 24 hours” and “How dissatisfied or satisfied are the patients with their pain relief after the surgical procedure?”. The nurse questionnaire used the same 5 and 11 point scales as in the patient questionnaire (Appendix 1). Questions on demographic variables were also asked in the nurse questionnaire. The answer from an individual nurse questionnaire was paired with the answer from an individual patient questionnaire. The questionnaire was pilot tested with seven nurses.
One of the researchers (EI) with no affiliation to the hospital conducted the study and recruited the patients and nurses on the five surgical wards to ensure that patients understood that their responses would not influence the care they received. The patients were informed both verbally and in writing and had the option to decline participation. The questionnaire was given to a patient and collected after about 20–30 minutes by the researcher. When a patient agreed to complete the patient questionnaire, the nurse responsible for the individual patient at that time was asked by the researcher to answer the nurse questionnaire within 1–3 hours. The nurses had been informed both verbally and in writing about the procedures before the study and also had the option to decline participation.
The study was examined and approved by the research ethics committee of the Faculty of Health Sciences, Linköpings University.
Data were analysed using Statistica 1999 version (StatSoft, Tulsa, Oklahoma, USA). Descriptive statistics are presented as arithmetic means with standard deviations for clarity although the questionnaire is referred to as an ordinal scale. The Wilcoxon matched pairs test was used to test differences in assessments between patients and their nurses. The Mann-Whitney U test and χ2 test with Yates’ correction were used to test differences between subgroups of patients. A multiple linear regression model using the backward stepwise method was used to analyse the explanatory variables believed to be potentially important both for patients and nurses. Kruskall-Wallis ANOVA followed by the Mann-Whitney U test and Bonferroni correction were used to test differences between patients’ assessments in general surgery, orthopaedics, and gynaecology. p values below 0.05 were considered significant.
The response rate was 96% (n=200). Seven women and two men, all aged >70 years, did not want to participate. Reasons for non-participation included, for example, being too tired and simply not interested in answering the questionnaire. Two questionnaires were omitted in the analysis because too many items were missed. Of the remaining questionnaires (n=198), 160 were fully completed; 32 patients had missed one partial item and six patients had missed two or three items. This was compensated for by mean substitution (the mean for that item by other patients). The data used for paired assessments were not mean substituted.
The mean (SD) age of the patients was 62.1 (14.9) years (range 20–87) and 120 (61%) were women. The mean (SD) age of the women was 61.9 (15.3) years (range 20–87) and of the men was 62.8 (14.3) years (range 21–84). The patients were undergoing a variety of surgical procedures in orthopaedics, gynaecology, and general surgery (table 1⇓). They had spent a mean of about 40 hours on the wards when answering the questionnaire. Sixty one patients (31%) received epidural analgesia for postoperative pain relief and only three received no postoperative analgesia. Thirty six patients (18%) asked the researcher to read the questions and complete their answers.
The mean (SD) score for the Strategic and Clinical Quality Indicators in Postoperative Pain Management (scale range 14–70) was 58.6 (8.9). The mean scores for the individual items were between 2.8 and 4.7 (scale range 1–5), and the percentage of patients who scored an item as 1 or 2 (disagreement) varied from 0.5% to 52.0% (table 2⇓). The mean scores for the subscales and complementary questions are shown in table 3⇓.
Sixty three of the 64 nurses agreed to participate in the study. The number of completed nurse questionnaires paired with an individual patient was 196 of 198 possible questionnaires (99%). Two questionnaires were not completed because one nurse who was in charge did not want to participate, and one questionnaire was forgotten. Of the 196 questionnaires, 180 were fully completed, 11 missed one partial item, and five missed two items.
The mean (SD) age of the nurses was 38.4 (8.9) years (range 23–57), and 59 (94%) were female. They had been registered for a mean (SD) of 13 (9.4) years (range 0.5–33) and had worked in surgical nursing care for 12 (8.7) years (range 0.5–33). Twenty nurses (32%) had university training in pain management. In 97% of the 196 possible times a questionnaire was completed, the “paired” nurse had been responsible for the patient during the previous 4 hours. Also, in 86% of the possible times, the nurses had been responsible for the “paired” patient at least one working shift (about 8 hours) during the past 24 hours. Each nurse answered a mean (SD) of 3 (2.3) questionnaires (range 1–11) during the 4 months during which data were collected.
The mean (SD) score on the Strategic and Clinical Quality Indicators in Postoperative Pain Management instrument (scale range 12–60) was 48.1 (6.2). The mean scores for the individual items were between 3.3 and 4.8 (scale range 1–5) and the percentage of nurses who scored an item as 1 or 2 (disagreement) varied from 0.0% to 34.8% (table 2⇑). The mean scores for the subscales and complementary questions are shown in table 3⇑.
The total scale was used as an outcome variable in a multiple linear regression model. The explanatory variables were “number of years working in surgical nursing care” and “university training in pain management or not”. The total scale (β=0.25, p<0.001) was associated with “numbers of years working in surgical nursing care”—that is, nurses with more years reported higher scores.
Comparison of patients’ and nurses’ scores
There were differences between patient and nurse assessments on the environment subscale, with patients having higher scores. The patients also scored higher than nurses on the complementary dissatisfied/satisfied question. Patients had higher scores than nurses on the question about the worst level of pain intensity, which shows that patients experienced greater pain than nurses thought (table 3⇑).
Comparison of patient subgroups
Patients aged ≥70 years had higher mean scores on the environment subscale than those aged <70 years. Women had lower scores than men on the action subscale and higher pain intensity scores on the question relating to the lowest pain level. Patients with epidural analgesia had higher mean scores than those without epidural analgesia on the total scale and the action subscale, but a lower score on the environment subscale. Forty two patients (24%) reported more pain than they expected. These patients had lower scores than those who reported no more pain than they expected on the total scale and on the four subscales. Moreover, they experienced higher pain levels on the worst, lowest, and pain now subscales and were less satisfied with overall pain relief. These differences are shown in table 4⇓.
The total scale and the four subscales were used as outcome variables (dependent) in the multiple linear regression model. The explanatory variables (independent) were sex, age, epidural analgesia, and more pain than expected. The total scale was associated with the explanatory variable more pain than expected (β=0.27, p<0.001)—that is, reports of more pain than expected had lower scores. The communication, trust, and environment subscales were associated with more pain than expected (β=0.24, p<0.01; β=0.21, p<0.01; and β=0.26, p<0.001, respectively). The environment subscale was also related to sex (β=–0.16, p<0.05) and age (β=0.23, p<0.01)—that is, women and older patients had higher scores. The action subscale was related to epidural analgesia (β=–0.22, p<0.01) and sex (β=0.17, p<0.05)—that is, those receiving epidural analgesia and men had higher scores.
Patient assessments in different departments
Patients in orthopaedics scored higher on the total scale and on the four subscales than those in general surgery, but there were no differences in worst, right now, and lowest pain levels or degree of dissatisfaction/satisfaction. Patients in gynaecology scored higher on two subscales than those in general surgery, and they were more satisfied with overall pain relief (table 5⇓). The gynaecological patients were younger than the orthopaedic patients (p<0.001) and, naturally, there were more women than in both general surgery and orthopaedics (p<0.001). No differences were found between the numbers of patients with more pain than they expected and those receiving epidural analgesia in the different wards.
This study has shown differences between patient and nurse assessments of quality of care in postoperative pain management. Differences were also found between subgroups of patients and in different clinical settings.
An instrument which reports data on both item and subscale levels has considerable advantages. The subscales map out areas which could be used for quality improvement on a broader basis such as the departmental level. The different items give more detail for staff working with the patients and may point out aspects of care which need to be further developed. Another questionnaire used for quality development in postoperative pain management is the American Pain Society (APS) Patient Outcome Questionnaire16 which has been used for patients with acute and cancer pain. The psychometric properties of this instrument have not been evaluated. The APS questionnaire has been modified in several ways in different studies and the results are reported on the item level.17–,19 However, the Strategic and Clinical Quality Indicators in Postoperative Pain Management was especially developed for postoperative pain management and tested for psychometric properties. It should give valuable baseline data which can be used to identify areas for quality improvement to be followed over time by patients and staff at different levels in the organisation.
The instrument items are all positively worded—that is, a high score is always good. This has not been discussed in previous studies of the instrument8,13 but could present a problem of “response set” bias—that is, respondents recognise that a high score is always good and respond instinctively without reflecting appropriately on what is being asked. On the other hand, the instrument is short with only 14 items which, hopefully, are clear and easy to understand.
The present study was performed on two or three days per week. The inclusion criteria, the second postoperative day and elective surgery excluded the other week days. Patients whose second postoperative day in the wards fell on a Saturday or Sunday were few and were not included. This exclusion did not omit different surgeons or surgical procedures.
The best time to apply a patient questionnaire to evaluate postoperative pain management is debatable. We used it on the second postoperative day, but the APS questionnaire16 has been used on different days including at the time of discharge and on the second postoperative day.18–,20 To be able to compare the results it is important to use the same time for the patients or for a subgroup of patients. The best time for different surgical procedures needs to be evaluated further. It has been reported that a patient’s pain intensity level is most severe about 37 hours after surgery.4 If a response is given near this time a more critical view of the pain management might be expected than on discharge when there is likely to be less pain.
What score or threshold constitutes “high” quality of care? According to the Joint Commission (page 107),1 a threshold is “a level or point at which the results of data collection in monitoring and evaluation trigger intensive evaluation of a particular important aspect of care to determine whether an actual problem or opportunity for improvement exists”. As the indicators studied have been derived from important aspects of care in the department, the threshold will, of course, be high, leaving only a small percentage of non-conformances because of uncontrolled factors. A baseline threshold is defined as a starting point for the process of continuous improvement which should be compared with the ideal threshold that has been established by, for example, a panel of experts.1
In the present study the items used in the instrument have been validated both by nurses with special knowledge in pain management and by clinical nurses as: essential for the quality of care, realistic to carry out, and possible for nurses to influence management.10,11 It is therefore reasonable to suggest high thresholds. A mean score for each item which exceeds 4.5 (scale range 1–5) could, for example, be considered as the level required for high quality of care. If this mean score is not obtained, it would be important to scrutinise the quality of care and suggest quality improvement efforts. The number of patients who scored an item as 1 or 2 (disagreement) could be considered as an area for improvement. Looking at the items with a mean score exceeding 4.5, the percentage of patients scoring 1 or 2 was 0.5–3%. This could be regarded as a reasonably small percentage of non-conformances. At the ward level it is valuable to examine the individual items in quality development while the total scale and subscales are important at the departmental and hospital levels.
The desired mean score for the total scale should therefore exceed 63.0 (14 items × 4.5), for the communication and environment subscales 13.5 (3 items × 4.5), and for the action and trust subscales 18.0 (4 items × 4.5). These desired mean scores could be compared with the baseline data. The desired mean scores for the total scale and the three subscales (communication, action, and environment) were not reached by the patients in our study, which indicates that an intensive evaluation should be performed of the different subscales and/or their items to determine whether an actual problem or opportunity for improvement exists. Looking at the number of patients scoring 1 or 2 on the subscales, there are obviously possibilities for improvement, especially on the action subscale. Setting thresholds as absolutes is, of course, not recommended although there is always variability.1 Threshold parameters which, in our study, should define the lower limit of acceptance for the result could be discussed and set statistically in retrospective analysis when further studies have been performed. When evaluating the different departments according to the desired mean scores, the orthopaedic and gynaecology wards reached the level for high quality of care on the trust and environment subscales. This indicates that an absolute score for high quality of care should not be set even though, for example, no significant difference was found in the scores for the trust subscale between general surgery and gynaecology, but the desired mean score was reached only in gynaecology.
The threshold or cut off level needs further consideration. Future projects could, for example, involve patients more centrally to elicit how important they perceive the statements to be. It is, however, important to combine this with professional perspectives on how to avoid complications postoperatively.
Not surprisingly, patients who reported more pain than expected had lower scores on the total scale and on the communication, trust, and environment subscales. Those who received epidural analgesia scored higher on the action subscale, possibly because of the special and distinct routines explicit for these patients such as using a pain assessment instrument. The differences between groups of patients must be considered when further evaluating quality of care and quality improvement efforts.
No difference was found in the overall satisfaction scores between patients in general surgery and those in orthopaedics, but there were differences between these two groups of patients in the scores on the total scale and the four subscales. The question on overall satisfaction is inadequate to indicate necessary quality improvements and differences between departments and groups of patients; many patients who experience severe pain also report satisfaction with the overall pain management.2,17,21 The pattern of pain relief, rather than pain severity itself, has been used as a critical determinant of satisfaction. Patients are satisfied even though they are in pain because they experience a commonly expected peak.22 A study by Pellino and Ward21 found that the perception of having control over pain was highly related to satisfaction, even in the presence of severe pain. The validity of patient satisfaction as an optimal outcome variable in quality assurance was also questioned in a study by Svensson et al.23
The nurses’ scores did not reach the earlier suggested desired threshold for high quality of care on the three possible subscales, which may indicate their awareness of non-compliance with a general standard for high quality care in postoperative pain management. The reasons for this may, of course, vary. However, looking at the individual items, those concerning enough nurses on duty and whether nurses are knowledgeable scored >4.0, which clearly were not major problems for the nurses who took part in the study. Scores on the total scale were higher for those who had worked longer on surgical wards, presumably because they had increased knowledge and confidence.
Comparison of patients’ and nurses’ assessments
Patients’ scores for the environment subscale were higher than those of nurses. It is not surprising that nurses gave a lower assessment than patients to “enough nurses on duty” and “the rooms are pleasant”. No difference was found between nurses’ and patients’ scores on the action subscale, but the percentage of nurses and of patients scoring 1 or 2 was the highest for this subscale. Only a small percentage of both nurses and patients scored 1 or 2 on the trust subscale. Possibilities for improving the quality of care from the nurses’ perspective could be identified and were in agreement with the patients’ assessments. It is clearly an advantage if both patients and nurses have similar opinions about the quality of care delivered and facilitate quality improvements.
The patients in the present study reported a higher pain intensity level on “worst pain” than the nurses thought they had experienced. Several studies have reported a discrepancy between nurse and patient assessments of patients’ pain intensity level, both at rest and when coughing, and the discrepancy increased with the level of pain.5,7,24,25 The reason nurses assess pain intensity levels lower may be that they subconsciously know they have no time to act or they feel inadequate on how to act. The nurses in the present study, however, scored the complementary dissatisfied/satisfied question lower than the patients, which might indicate that the nurses were aware that pain management could be better from their professional perspective. A study by Lövgren et al26 of satisfaction with the general quality of care also found that the staff perceived that the patients were less satisfied than they actually were. The authors suggested that the staff probably based their assessments on professional experience, while the patients expressed personal non-professional opinions. A study of quality of care by Larsson et al27 showed that elderly home care users evaluated most care components more favourably than did their caregivers.
A new instrument, the Strategic and Clinical Quality Indicators in Postoperative Pain Management, with complementary questions was used to assess quality of care.
Areas for improvement in postoperative pain management were identified—for example, pain assessment at rest and when moving and information on individual types of pain treatment.
There were differences in patients’ and nurses’ assessments of the quality of care in postoperative pain management—for example, worst possible pain intensity and general satisfaction were both rated higher by patients.
The threshold for “high” quality of care in postoperative pain management can be considered from different perspectives.
The Health Research Council in Southeast Sweden supported this research.