Article Text
PDF
Abstract
Objectives This study presents results from an electronic survey among paediatric department employees, addressing employees' attitudes and use of results from a national parent experience survey carried out in 2005.
Methods Electronic questionnaire survey of employees from each of the 20 paediatric departments included in the national survey, with a response rate of 87%.
Results The employees had favourable opinions of user experience surveys, and the results from the national survey were well known among both managers and other personnel. User experience surveys were considered important, and 56% reported that they had implemented improvement actions addressing problems identified in the national survey. Managers reported more often than staff without managerial responsibility that the results had been informally discussed, and that the survey was useful for their own department. Department leaders were more positive to the usefulness of the survey than non-leaders. Significant differences in attitudes were found between physicians and other health personnel.
Conclusion Employees in the paediatric departments were positive to user experience surveys, and the surveys have a potential to be actively used in quality improvement actions. Effects of the quality improvement initiatives should be assessed in future parent experience surveys.
- Quality improvement
- patient satisfaction
- electronic questionnaire
- quality of healthcare
- paediatrics
- continuous quality improvement
- healthcare quality improvement
- patient-centred
- quality of care
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
- Quality improvement
- patient satisfaction
- electronic questionnaire
- quality of healthcare
- paediatrics
- continuous quality improvement
- healthcare quality improvement
- patient-centred
- quality of care
Introduction
There has been increasing emphasis on the use of patient satisfaction surveys in health services to assess quality of care. The purpose of these surveys is often related to quality improvement. However, a range of methodological challenges threaten the value of using survey data, such as psychometric properties of the measurement instruments,1 non-response2 and case-mix.3 ,4 Many studies have focused on methodological issues, but few have investigated their active use and utility in quality improvement efforts at the organisational level.5–7
A French study found positive attitudes towards patient experience surveys among employees at a University hospital, but the results were not systematically used in quality improvement.6 Another study found organisational, professional and data-related barriers for active use of patient survey data, and concluded that better use requires the development of cultures that support patient-centred care, quality-improvement capacity, professional receptiveness and leadership, and technical expertise with these data.5 Corresponding conclusions were made in a third study.7
Due to the paucity of studies addressing this issue, we carried out a follow-up survey among health personnel at specialised children departments included in a national parent experience survey. The aim of the study was to explore employees' attitudes to user experience surveys, their opinions on the usefulness of the national survey as well as their reported use of the results.
Method
The head of department at each of the 20 specialised children departments included in the national parent experience survey in 20058 was asked to choose five employees from their own department. Selection was based on the following criteria (ranked by importance): departmental leader, employees with a special role in quality improvement, other physicians, nurses. Contact information for the sample was sent to us by each head of department.
The survey was conducted electronically from June to October 2007. Non-respondents were sent up to four reminders.
Questionnaire
The questionnaire was based on a previous study,6 with modifications conducted by experts on survey methodology to capture issues specifically relevant to this study.
The questionnaire comprised questions concerning attitudes to user involvement and user experience surveys, the national parent experience survey in 2005 and employees' opinions on the use and usefulness of the national survey. Most questions had a five-point response format that ranged from ‘not at all’ to ‘a very large extent.’ Results on 16 of the 18 closed-ended questions and one of the six open-ended questions are reported.
Analysis
Statistical analyses were carried out using SPSS (15.0; SPSS, Chicago, Illinois). Descriptive statistics are presented for the closed-ended questions about attitudes, use and usefulness. The mean and SD for department leaders, head of sections and employees without a management position are reported for questions with a five-point response format. One-way analysis of variance (ANOVA) with Bonferroni post hoc correction was used to test differences between groups. Remaining questions are reported as percentages answering ‘yes,’ and the χ2 test was applied to test differences. The questions measuring attitude were included in an exploratory factor analysis, and the Cronbach α coefficient was applied to evaluate the internal consistency of the scale.
The association between socio-demographic and professional characteristics and attitudes as well as a single item addressing the usefulness of the survey was assessed by bivariate and multivariate linear regression analyses. An exploratory analysis was chosen due to the scarceness of studies. The association between the two dependent variables and each of the independent was explored by bivariate regression models. Independent variables with p<0.1 on at least one of the dependent variables were included in multivariate regression models.
Content analysis was applied on responses to the open-ended question about implemented improvement actions. The responses were categorised into major topics, and a content analysis of each topic was conducted to identify the most important themes.9
Results
Response rate and characteristics of respondents
Nineteen of 20 children departments from the national parent survey participated in the follow-up survey. One hundred employees were included and posted the questionnaire. Nine never received the email and were excluded. Completed questionnaires were returned by 79 (87%). Twelve respondents filled out paper questionnaires.
Seventy-one per cent were female, the mean age was 46 years, and the mean work experience was 14 years. Fifty-three per cent were trained nurses, and 38% were physicians. Sixty-two per cent had a management position; 23% were department leaders, and the remainder were a head of a section or had other management positions below the department level. Forty per cent worked at a district general hospital, 38% at a University hospital and the remainder at a local hospital.
Main results
All attitude items regarding user involvement were rated positively. On a scale from 1 to 5, where 5 represent the most positive attitude, the mean score was around 4 (table 1). The importance of patient experience surveys had the highest total score of the items concerning user experience surveys (4.16), while the question addressing the potential effect on medical quality had the lowest score (3.15). No significant differences were found between department leaders, head of sections or employees without a management position on the attitude items.
Descriptive statistics for questions with five-point response format*, grouped by type of position within the department
Respondents attached a lower value to the assessment of the usefulness of the national survey (2.89) and to the question concerning informal discussion of the results (2.73) than to the attitude items (table 1). Department leaders had significantly higher scores than the other groups as concerns informal discussion (F value: 7.62, p=0.001). Department leaders also had a more positive evaluation of the usefulness compared with employees without a management position (F value: 3.86, p=0.026).
Most respondents (86%) had been informed about the results for their own department, and 78% reported that the results had been discussed (table 2). Almost 20% believed that the national survey had changed the way health personnel behave towards patients. Many (65%) had knowledge of the report from the survey, and 56% reported that improvement actions had been implemented as a consequence of the survey. Department leaders and other leaders gave more positive responses than employees without a management position on all questions in table 2; however, χ2 tests are not reported because the sample size requirement for the test was not satisfied; the expected frequency for many of the cells was less than 5 and could not be interpreted.
Percentage ‘yes’ for questions with ‘yes’ and ‘no’ response categories, grouped by type of position within the department
The nine questions measuring attitude were included in an exploratory factor analysis, and a unidimensional scale was identified. One of the questions was excluded due to low factor loading and low item–total correlation. The final scale had a Cronbach α coefficient (α=0.89) well above the recommended value of 0.7;10 ,11 item-total correlations were also satisfactory (0.46–0.77).
Table 3 shows that several variables have a significant bivariate association with either the attitude scale or the question addressing usefulness (p<0.1). However, the multivariate regressions (table 4) identified only two significant associations; physicians were less positive than others on items comprising the attitude scale (p=0.01), and department leaders were more positive to the usefulness of the survey than employees without a management position (p=0.042).
Bivariate association between independent variables and the attitude scale and the question about usefulness
Multivariate regression models on the association between independent variables and the attitude scale and the question about usefulness
Forty-three employees reported that the department had implemented improvement actions based on results from the national survey, and 36 respondents described the actions more specifically. Major topics were information, role clarification, continuity and routines. Role clarification was related to relationships between employees and parents but also within the department. Predictability was underlined as a main target of efforts addressing the relation and dynamics between health personnel and parents. Continuity was mainly related to health personnel continuity for parents during the hospital stay. The comments also showed that the children departments endeavour to optimise routines to meet the needs for improvement of standardisation.
Discussion
This national study shows that health personnel in paediatric wards report active use of results from a parent experience survey in quality-improvement work at the ward level. We did not intend to assess the national surveys' effect on quality, but sought the opinions of important target groups on the follow-up of the survey. To our knowledge, this is the first study focussing on the use and usefulness of a national parent experience survey conducted at hospital departments for children.
Hospital management have been emphasised as most responsive to data from quality-measurement studies, especially top management, and a common effect is implementation of quality-improvement initiatives.12 ,13 The effect of these initiatives is debated and beyond the scope of our study, but we did find that leaders gave the most favourable assessment of effect and usefulness. Attitudes were affected by professional background, while assessment of usefulness depended on the type of position. These findings are confirmed by other studies.6 ,14 A recent study illustrates the importance of this issue, showing that the personal involvement of the department head played the key role in the continuation of the quality-management programme.15 Another study also concluded that it seemed especially difficult to engage medical doctors.16
The study sample was constructed to measure attitudes among leaders and employees involved in quality improvement, because those groups are core personnel in implementing improvement projects based on quality measurement. The survey was national and included all hospitals in Norway with a children department. The response rate was high, implying that potential effects related to non-response are of minor concern. Therefore, the results can be expected to be representative at the national level for the groups included in the study sample. Obviously, testing the effect of the national survey on all personnel at hospital children departments would call for another design. The generalisability of our results to other settings is uncertain. We built our study on the study by Boyer et al,6 the latter being conducted at a French teaching hospital. Our findings resemble the main findings in the study by Boyer et al,6 but the poor knowledge base regarding this topic warrants more research to validate the findings in other settings. We are currently planning a similar study within mental healthcare.
The inclusion of attitudinal questions means that some of the results are not very specific. The reason for including attitudinal questions is twofold. First, we wanted to base and compare our study with the study by Boyer et al,6 and they included several attitudinal questions. Second, the qualitative study by Davies & Cleary5 identified professional barriers as one important barrier to the use of patient survey results in quality improvement. Negative attitudes might impede the active use of survey results, so we wanted to assess the extent of this barrier in our study. Both our study and the study by Boyer et al6 show that health personnel had positive attitudes to patient surveys, indicating that the other barriers in the study by Davies & Cleary5 might be more important in these settings. However, future research might use fewer questions about attitudes making more room for questions with a more specific focus.
It is commonly assumed that feedback from patients on professional performance might lead to improvement activities and, accordingly, a change in behaviour towards the patients. Despite positive attitudes, only 18% reported that the results of the national survey had led to changes in the behaviour of health personnel. However, we believe that the main reason for this result is poor knowledge of the topic addressed by this question. To reduce social desirability bias, we have asked the respondent to assess behaviour change for health personnel in the department but overestimated the respondents' knowledge of this topic; 56 (74%) of the respondents answered that they did not know whether other health personnel had changed their behaviour. In future surveys, this topic should be addressed by improved questions including changes in one's own behaviour in addition to changes for other groups of health personnel.
The national survey found that certain specific aspects related to organisation and nursing were rated most negative.8 The quality-mprovement actions reported in the current study were closely related to these aspects, but we are unable to validate questionnaire responses because we lack other data on actual quality-improvement work at the departments. However, qualitative comments include detailed descriptions of a range of follow-up activities. Another methodological issue is related to social desirability bias: we collected data in a study assessing another of our own surveys. This might have affected responses, especially by increasing the pressure for giving social desirable answers. However, questions related to assessment of our national survey had the most negative responses (ie, usefulness), indicating little social desirability bias.
A major goal for user experience surveys is active use of results in quality-improvement work,17 but few studies explore this systematically, and results are contradictory. This study showed that our national parent experience survey has influenced quality work. Whether or not the initiatives and actions actually improve quality needs to be explored in future studies including new parent experience surveys.18
References
Footnotes
Funding Norwegian Knowledge Centre for the Health Services.
Competing interests None.
Ethics approval The Norwegian Social Science Data Services.
Provenance and peer review Not commissioned; externally peer reviewed.