Article Text
PDF
Abstract
Objective To measure patient safety culture in Belgian hospitals and to examine the homogeneous grouping of underlying safety culture dimensions.
Methods The Hospital Survey on Patient Safety Culture was distributed organisation-wide in 180 Belgian hospitals participating in the federal program on quality and safety between 2007 and 2009. Participating hospitals were invited to submit their data to a comparative database. Homogeneous groups of underlying safety culture dimensions were sought by hierarchical cluster analysis.
Results 90 acute, 42 psychiatric and 11 long-term care hospitals submitted their data for comparison to other hospitals. The benchmark database included 55 225 completed questionnaires (53.7% response rate). Overall dimensional scores were low, although scores were found to be higher for psychiatric and long-term care hospitals than for acute hospitals. The overall perception of patient safety was lower in French-speaking hospitals. Hierarchical clustering of dimensions resulted in two distinct clusters. Cluster I grouped supervisor/manager expectations and actions promoting safety, organisational learning–continuous improvement, teamwork within units and communication openness, while Cluster II included feedback and communication about error, overall perceptions of patient safety, non-punitive response to error, frequency of events reported, teamwork across units, handoffs and transitions, staffing and management support for patient safety.
Conclusion The nationwide safety culture assessment confirms the need for a long-term national initiative to improve patient safety culture and provides each hospital with a baseline patient safety culture profile to direct an intervention plan. The identification of clusters of safety culture dimensions indicates the need for a different approach and context towards the implementation of interventions aimed at improving the safety culture. Certain clusters require unit level improvements, whereas others demand a hospital-wide policy.
- Organisational culture
- benchmarking
- patient safety
- cluster analysis
- assessment
- health policy
- organization
- patient safety
- safety culture
Statistics from Altmetric.com
- Organisational culture
- benchmarking
- patient safety
- cluster analysis
- assessment
- health policy
- organization
- patient safety
- safety culture
Introduction
Patient safety is receiving growing attention in Belgium. A 5-year program (2007–2012) was launched to implement quality and patient safety initiatives in the acute, psychiatric and long-term care hospitals, with a yearly additional financing (annual budget of €6.8 million in 2007). In 2007, the federal contract was signed by 80% (n=164) of the hospitals, including 97 acute hospitals, 52 psychiatric hospitals and 15 long-term care hospitals. The Belgian government provides a framework for implementing quality and safety strategies with attention to structure (how care is organised), processes (what is done by healthcare providers) and outcome measurement (the healthcare results achieved), according to Donabedian's trilogy.1
One of the main priorities in the federal program is developing a culture of safety. “Safety culture refers to the beliefs, values and attitudes of patient safety shared by all members of the organisation. These shared values are reflected in the day to day operations of the organization”.2 Understanding safety culture is seen as a key component in improving patient safety in Belgian hospital settings. During the first program year (2007–2008), 158 hospitals completed a hospital-wide measurement of the safety culture using the Hospital Survey on Patient Safety Culture (HSPSC).3 During the second program year (2008–2009), 22 other hospitals entering the federal patient safety program assessed the safety culture. In total, 88% of the Belgian hospitals (180 out of 205 hospitals) applied the HSPSC to measure the hospital-wide safety culture. The federal government is planning to organise a second measurement in 2011 in order to track changes in patient safety culture over time and evaluate the impact of specific safety interventions.
The Belgian versions, manual, psychometric validation reports and instruments of the HSPSC are available in Dutch, French and German.4 Yet, the instrument is also highly recommended by the European Union Network for Patient safety (EUNetPaS) for internal use, though not for benchmarking.5
In many other countries, the HSPSC is used to measure safety culture, and previous research has shown that the instrument is psychometrically sound.6–9 The instrument has also been tested to determine the most appropriate level—individual, unit and hospital level—for interventions aimed at improving the culture of patient safety. The unit level appears to be the dominating level for the clustering of responses to the dimensions, which would confirm that the HSPSC measures group values of culture and not just individual attitudes.10 Previous Belgian research suggested differences between professional subgroups, although no representative conclusions could be made for the Belgian hospital sector.11
This study reports on a national aggregation of the data of the HSPSC within the Belgian hospitals and aims at providing each hospital a baseline score on 12 dimensions in order to set priorities and follow-up on the evolution of safety culture. In this way, the measurement of safety culture reflects a “snapshot” of the current state of safety culture within the hospitals.12
Objectives
The primary aim of the study was to measure patient safety culture in Belgian hospitals. In order to formulate actions for improvement, it is important for hospitals to assess their baseline scores for the existing safety culture and determine areas of priority.
This study describes the survey results of the acute, psychiatric and long-term care hospitals that voluntarily submitted their data for comparison to other hospitals. In addition, this study aims at examining (clustering) the underlying dimensions of patient safety culture. Results of these analyses can provide additional information on the common strengths or areas that need improvement.
A second nationwide survey and benchmarking are planned 3 years after the initial measurement to track changes in patient safety culture over time.4 The nationwide safety culture measurement already raised high awareness about the role of culture in Belgian hospitals and may in itself be regarded as a patient safety initiative.
Methods
Instrument
The HSPSC measures safety culture on 12 dimensions, including 10 safety dimensions and 2 outcome dimensions and is designed to measure staff perceptions on patient safety issues, medical errors and event reporting (table 1).3
Safety and outcome dimensions
The validation of the translation into Dutch and French was performed using the original validation strategy and included item analysis, exploratory factor analysis, confirmatory factor analysis, reliability analysis and analysis of the composite scores and intercorrelations.13 The reliability coefficients (Cronbach's α) of the 12 safety culture dimensions ranged from 0.57 to 0.85 for the Dutch version and between 0.52 and 0.87 for the French version, which is comparable to the original questionnaire. Frequency of events reported and staffing showed, respectively, the highest and lowest internal consistency (Online appendix A).
Data collection
The HSPSC was distributed organisation-wide in 180 (88%) Belgian hospitals participating in the federal program on quality and safety in 2007–2009. A first group of 158 hospitals initiated the safety culture assessment in 2007–2008, while 22 other hospitals started up 1 year later. Through a contract with the federal authorities, participating hospitals (in their first contract year) committed to measure safety culture within the entire organisation. A workshop was organised for the participating hospitals in which the objectives and the tools for conducting the safety culture measurement were explained. The measurement toolkit contained the validated version of the HSPSC (in Dutch and French) and a manual (protocol). The protocol was comparable to the original version and imposed a time plan of 13 weeks with the encouragement to sent two reminders to non-responders. Though, not all hospitals sent reminders. An MS Access-based instrument was designed to standardise data entry and automate the application of the exclusion criteria and analyses. Throughout the measurement period, technical assistance was available. Hospitals were free to distribute the survey electronically or paper-based. The questionnaire was distributed anonymously to all individuals working in direct or indirect interaction with patients. Participating hospitals were invited to submit their data to a research database created by Hasselt University, a neutral academic institute. The database is not accessible for the governmental authorities and was developed to allow hospitals to compare their data to other hospitals and to provide data to hospitals to facilitate internal assessment and learning in the patient safety improvement process.
Statistical analysis
Questionnaires were excluded in case an entire section was incomplete, fewer than half of the items throughout the survey were answered and all items were scored identically (as defined in the manual of the HSPSC questionnaire).
First, a mean dimensional score (range 1–5) was calculated on the individual level. Answers to negatively worded questions were reversed. Dimensional scores higher than three were considered as positive values towards patient safety. Based on these values, the dimensional scores were calculated on the hospital level (percentage positive values of all individuals). A percentage on an item was given on the total number of respondents for this specific item.
Based on the positive dimensional hospital scores, a hierarchical cluster analysis was conducted using a squared Euclidean distance measure to assess similarity/dissimilarity across variables. Ward's algorithm for hierarchical cluster analysis was selected because it minimises the heterogeneity of the clusters. Ward's method builds the hierarchy from the individual elements by progressively merging clusters in order to minimise the internal variance. In short, this method attempts to minimise the sum of squares of any two (hypothetical) clusters that can be formed at each step.
In addition, three other cluster algorithms—between-groups linkage, centroid clustering and median clustering—resulted in the same clusters confirming the robustness of the result found by Ward's method. Based on the cluster algorithm, a dendogram was generated for visual classification of similarity for grouping the underlying dimensions. In the dendogram, the dimensions were represented as nodes and the branches illustrated when the cluster method joined groups of dimensions. The length of the branch indicated the distance between groups of dimensions when they were joined.
All data were analysed confidentially. SPPS V.17 was used for all analyses.
Results
Characteristics of participating hospitals
Ninety acute, 42 psychiatric and 11 long-term care hospitals were interested in comparing their results to other hospitals. A comparative report was provided to each hospital, including its position on each dimension among other hospitals according to the type of hospital (acute, psychiatric and long-term care), to facilitate internal assessment and learning in the patient safety improvement process.
Characteristics of the participating hospitals are presented in table 2.
Hospital characteristics
Characteristics of respondents
Respondents' characteristics are set out in table 3, based on the respondents' answers to survey questions about their hospital work area, staff position, direct interaction with patients, professional experience and working time in the hospital. Generally, respondents working in psychiatric and long-term care hospitals indicated for work area, respectively, “psychiatry” and “revalidation”.
Respondent characteristics
The benchmark database consists of data of 55 238 respondents (53.7% response rate) who completed the survey. Dutch-speaking hospitals had a higher overall response rate (59.6%) than French-speaking hospitals (43.1%), given a high variability between hospitals. A similar response rate was observed among questionnaires distributed on paper (53.8%) in comparison with the questionnaires that were distributed electronically (53.2%). In total, 49 925 employees (56.7% response rate) and 5313 physicians (35.6% response rate) completed the survey.
Positive dimensional scores
Figure 1 provides the dimensional scores expressed in terms of per cent of positive response. Per cent positive reflects the percentage of positive responses (eg, Agree, Strongly agree) to positively worded items or negative response to negatively worded items.
Boxplots for positive dimensional scores (one acute hospital was both Dutch and French speaking and was calculated as a French speaking hospital).
Cluster analysis
The clustering of the positive dimensional scores for the acute and psychiatric hospitals is shown in figure 2 (using Ward's method with the Squared Euclidean Distance measure). We refrained from clustering the dimensional scores of the long-term care hospitals because of the low number of participating hospitals (n=11).
Dendogram from the hierarchical cluster analysis of dimensions (acute and psychiatric hospitals).
For acute hospitals, clustering identified two distinct groups:
Cluster I: including the dimensions communication openness (dim 4), supervisor/manager expectations and actions promoting safety (dim 1), organisational learning–continuous improvement (dim 2) and teamwork within units (dim 3). Within this cluster, the distance between these four dimensions was small, with the smallest distance between dimension 1 and 4.
Cluster II: including the dimensions feedback and communication about error (dim 5), overall perceptions of patient safety (dim 11), non-punitive response to error (dim 6), frequency of events reported (dim 12), teamwork across units (dim 9), handoffs and transitions (dim 10), staffing (dim 7) and management support for patient safety (dim 8).
Within this cluster, two subclusters can be distinguished:
Dimensions feedback and communication about error and overall perceptions of patient safety;
Dimensions non-punitive response to error, frequency of events reported, teamwork across units, handoffs and transitions and management support for patient safety and, at a slightly larger distance, the dimension staffing.
For psychiatric hospitals, a similar structure was found (figure 2), except communication openness (dim 4) clustered within cluster II.
An additional test with small sample sizes of variables (10%) yielded the same results. Other clustering methods as between-groups linkage, centroid clustering and median clustering confirmed the robustness of the results (Online appendix B).
Discussion
This report presents the results of a national patient safety culture assessment, including results from 55 238 respondents (53.7% response rate) working in 143 Belgian hospitals.
Generally, areas teamwork within units, supervisor/manager expectations and actions promoting safety, organisational learning—continuous improvement and communication openness were dimensions that emerged as areas of strength. Handoffs and transitions, staffing, management support for patient safety, non-punitive response to error and teamwork across units showed potential for improvement.
Positive dimensional scores were higher for psychiatric and long-term care hospitals than for acute hospitals, suggesting that patient safety is more encouraged within these settings. Another explanation could be that patient safety is more often at danger in acute hospitals—inherent to the more complex tasks that are performed in this setting—resulting in more frequent witnessing of unsafe patient care and a more negative evaluation of safety by the healthcare professionals working in acute hospitals.
We found that perspectives involving organisational learning and continuous improvement, staffing and teamwork within units were more positive in French-speaking than in Dutch-speaking hospitals. However, on all other dimensions, we found in the French-speaking part a lower percentage of positive answers, with the lowest values on both outcome dimensions. Caution must be taken in the interpretation of these results, since the number of participating French-speaking hospitals was lower (less than half of the Flemish hospitals) and so were the response rates of the employees and physicians. Moreover, the response rate in the Dutch-speaking hospitals might be higher because of the earlier local patient safety initiatives, which might have raised the experience and awareness of patient safety in these settings.
Hierarchical clustering of patient safety dimensions suggested within acute hospitals clear clustering schemes of the dimensions related to teamwork within units, communication openness and learning. Results show an important role of the supervisor of the unit when working on these dimensions. Initiatives to improve these dimensions should be focused at the level of the individual unit.
In our study, staffing clustered at greater distance from the other dimensions. Our findings are in line with the results of studies investigating the multilevel psychometric properties of the survey, which suggest that staffing falls slightly below cut-offs in a number of areas.6 ,9 Also, staffing seems to be more related to teamwork between different units and handoffs, rather than teamwork within units, meaning that staffing cannot be allocated to differences in perceptions between units. Earlier research found a significant clustering at the hospital level of the dimensions feedback about and learning from error, teamwork across hospital units and non-punitive response to error.10 We found that these dimensions were related to frequency of events reported, handoffs and transitions and management support for patient safety. These dimensions are of importance at the hospital level, for instance when setting up a centralised incident reporting system.
One of the strengths in our study was that hospitals were asked to organise a hospital-wide safety culture assessment. We succeeded to collect responses of 4883 medical staff members and 50 342 employees. Yet, the nationwide safety culture measurement raised high awareness about the role of culture in Belgian hospitals and may in itself be regarded as a patient safety initiative.
Comparisons between our results and the AHRQ 2011 User Comparative Database Report,14 providing results from 472 397 hospital staff in 1032 American hospitals, show that in our study, the overall response rate was higher, with a remarkable higher response from the medical staff. Reminders were an important driver in the survey to get a satisfactory response rate. We refrained from comparing dimensional scores to other countries, since there are too many national or healthcare-specific differences that limit any useful comparison. In addition, there are too many differences in respondents' characteristics between studies, as our study mainly focused on healthcare providers who work in direct interaction with patients (85.7%). What is more, we applied a different method in calculating positive dimensional scores. In our study, data were first used to calculate a mean dimensional score (range 1–5) on the individual level. Based on these individual scores, the percentage of positive response was calculated on the hospital level. In the original method, the average per cent positive scores were calculated by averaging composite-level per cent positive scores across all hospitals in the database as well as averaging item-level per cent positive scores across hospitals. A disadvantage of this method, since the per cent positive is displayed as an overall average, is that scores from each hospital are weighted equally in their contribution to the calculation of the average.14
There are several limitations to our study
First, hospitals were not randomly selected. The database only included data of hospitals that voluntarily submitted their data for comparison and did not represent a randomly selected sample of all 205 Belgian hospitals. However, 79.6% of the acute, 61.8% of the psychiatric and 55% of the long-term care hospitals were included in the analysis. Overall, the characteristics of the included hospitals are fairly consistent with the distribution of all Belgian hospitals.15
Second, hospitals used a different survey method (paper, electronic or mixed-mode) and not all of the hospitals sent reminders, which could have lead to differences in response rate. Though, hospitals were instructed to conduct the survey through several workshops. Technical assistance was available if necessary.
Third, the data hospitals submitted were cleaned for out-of-range values (eg, invalid response values due to data entry errors) and blank records (where responses to all survey items were missing). In addition, some logic checks were made. Otherwise, data were presented as submitted. No additional attempts were made to verify the accuracy of the data submitted.14
Fourth, an important disadvantage of cluster analysis is that once a score is assigned to a cluster, it cannot be assigned to another one. Some scores may have more than one significant property or fall on the edge of two clusters.
Finally, we recognise the limitations of this quantitative approach of safety culture, measuring group values, perceptions and attitudes on predefined dimensions, which might underexpose other important layers of safety culture.16 Alongside repeated measurement of patient safety culture, additional qualitative research, such as focus groups, staff interviews or observations might highlight important dimensions that are more related with cultural dynamics and cultural change.3 ,11 An interesting area warranting further research lies in understanding the perception of healthcare leaders towards patient safety.17–19 Yet, a great polarisation is found in the management views on patient safety,17 but it is not clear how leadership influences organisational culture.
On the other hand, there is still limited evidence on a quantitative relationship between safety culture and outcomes measures of safety that apply to the entire hospital.20 ,21 Therefore, future research is needed to understand how improvement strategies influence patient safety culture and how safety culture assessment can be related to outcome measurement.
Conclusion
Clearly, healthcare organisations are interested in the potential for evaluating, benchmarking and improving safety culture measurement. In this study, we presented aggregated results of a nationwide survey on patient safety culture. In addition, we investigated clustering of the underlying dimensions of the HSPSC. Our results suggest a different approach towards certain clusters of dimensions and, on the other hand, confirm the robust composition of the survey towards the different dimensions.
Our recommendations for healthcare managers concern that interventions aimed at communication openness and teamwork within units should be handled decentralised at the unit level. Error management, transitions and staffing require a more centralised approach at the hospital level. Repeated measurement after several years can track evolution in these dimensions.
Further research should be based on the combination of both quantitative and qualitative approaches in the assessment of safety culture. Greater attention must be paid at the hospital management view of patient safety in order to evaluate the organisational readiness towards patient safety strategies. An important shortcoming in the HSPSC lies in the fact that some sections, such as work area and staff position, are not fully applicable in psychiatric and long-term care facilities. We recommend refinement of these categories in order to compare and understand possible differences in dimensional scores.
Acknowledgments
We are grateful to all the Belgian hospitals who took the time and effort to participate in this study.
References
Footnotes
Additional appendices are published online only. To view these files please visit the journal online (http://dx.doi.org/10.1136/bmjqs-2011-051607).
Funding This research was funded by Limburg Sterk Merk, Universiteitslaan 1, 3500 Hasselt, Belgium. The participating hospitals were partially funded by the Federal government program on quality and safety in healthcare.
Competing interests None to declare.
Provenance and peer review Not commissioned; externally peer reviewed.