Several studies in various countries have shown that a substantial number of patients suffer from adverse events in hospitals (Zegers M, De Bruijne MC, Wagner C, et al. personal communication, 2008).1^–8 Hospitals are becoming ever more conscious of the fact that there is much to improve in the safety of their patients, and consequently many hospitals have been implementing interventions to reduce the amount of adverse events. However, as in other high risk industries—such as chemical process industry, aviation and nuclear power—it is believed that to improve patient safety in healthcare, hospitals have to create a safety culture among their staff alongside structural interventions. Reform of organisational structures, clinical training, guidelines and information technology are not sufficient when achieving good quality and patient safety. The culture of an organisation consists of the shared norms, values, behaviour patterns, rituals and traditions of its employees.9 Safety culture is an aspect of the organisational culture. A positive safety culture guides the many discretionary behaviours of healthcare professionals toward viewing patient safety as one of their highest priorities.10

Several instruments are available to make an assessment of the safety culture in hospitals.1112 One is The Hospital Survey on Patient Safety Culture (HSOPS).13 It aims to measure multiple dimensions of patient safety culture in hospitals. Previous research has shown that the psychometric properties of the HSOPS are good.1113 The questionnaire is being used in several countries. It has been translated into Dutch for use in The Netherlands. The factor structure and psychometric properties of the Dutch translation (COMPaZ) have been described earlier.14

Like many other safety culture questionnaires, the HSOPS has to be administered individually to employees at a hospital or a hospital unit. However, the aim of the questionnaire is to measure the group culture within the hospital or unit. The assumption that the HSOPS measures culture and not just individual attitudes has not been examined before. It can be tested by analysing whether individual responses cluster within units or within hospitals. By examining the clustering of individual responses by means of multilevel analysis, one can identify contextual phenomena.15 Clustering of individual responses would support the validity of the instrument.

Moreover, when healthcare organisations are planning to improve patient safety culture, they have to know the most appropriate level at which to direct their interventions. At first sight, a centralised (hospital level) approach might seem least time- and money-consuming, but in reality it is very possible that a decentralised approach (unit level) is the most efficient way of improvement for some or even all aspects of patient safety culture. Unit-level interventions can be tailored to the specific needs of a unit, while only a small part of the units within a hospital will gain from interventions implemented at hospital level.

Clustering of staff responses at unit or at hospital level indicates which dimensions of patient safety culture belong to and can be addressed at which organisational level.

In this study, we examine two main research questions:

1. Does the HSOPS measure group culture or only individual attitudes?

2. Which dimensions of the HSOPS reflect hospital characteristics, and which reflect unit characteristics?

With respect to the first research question, we expect all 11 dimensions to cluster significantly at either the unit level or hospital level (hypothesis 1). Some dimensions will show more clustering at hospital level than others. When considering the contents of the dimensions, three of the 11 dimensions seem to reflect hospital wide characteristics: Teamwork across hospital units, Adequate staffing and Hospital management support. We hypothesise that these dimensions will show significant clustering at hospital level, besides clustering at unit level (hypothesis 2a). The other eight dimensions seem to be specific features of a unit: Feedback about & learning from error, Overall perceptions of safety, Frequency of event reporting, Supervisor/manager expectations & actions promoting safety, Teamwork within units, Non-punitive response to error, Smooth transitions and Openness of communication. With respect to these dimensions, people working in the same unit are expected to be more similar to each other than to people working in different units, due to the context in which they work. There are no reasons to assume that different units within one hospital will show much similarity on these themes. Thus, we hypothesise that clustering is mainly within units for these dimensions (hypothesis 2b).

METHODS

Data collection

The questionnaire was administered in 19 hospitals across The Netherlands; eight hospitals in May–June 2005 and 11 in May–June 2006. Hospitals differed by teaching status: nine general hospitals, nine teaching hospitals and one university hospital. A total number of 87 hospital units participated in the study. Units and hospitals were not randomly selected. Units that participated were about to introduce an incident reporting system at their unit and wanted to assess their patient safety culture prior to the implementation of the new system. In each unit, a random sample of about 30 healthcare providers was drawn, depending on unit size.

The response rates of 67 of the 87 units were scored. Because, in 20 units, the distribution of the questionnaire was not carried out by the researchers, there was no reliable information about the number of people having received a questionnaire in these units. Therefore, it was not possible to calculate the response rates for these units.

Questionnaire

We translated the HSOPS into Dutch using forward and backward translation to check the quality of the translation. Exploratory factor analysis has demonstrated that the questionnaire contains 11 dimensions: Feedback about & learning from error, Overall perceptions of safety, Teamwork across hospital units, Frequency of event reporting, Supervisor/manager expectations & actions promoting safety, Teamwork within units, Non-punitive response to error, Smooth transitions, Openness of communication, Adequate staffing and Hospital management support.14

The questionnaire consists of 51 items, including background variables outlining the participant’s professional group (registered nurse, physician, secretary, manager, etc), unit type, number of years of employment and number of hours a week at the unit. A total of 44 items relate to patient safety culture. Respondents are asked to rate each item on a five-point scale of agreement (strongly disagree, disagree, neutral, agree and strongly agree) or frequency (never, rarely, sometimes, most of the time, always). The number of items per dimension ranged from 2 to 6, and all items within the 11 dimensions were normally distributed. The internal consistency (Cronbach alpha) of the dimensions ranged from 0.58 to 0.79. The items and factor structure of the Dutch questionnaire are only slightly different from the original HSOPS. The original questionnaire has 12 dimensions, but in the Dutch factor analysis two dimensions converged. The composition of the other dimensions was very similar to that of the original questionnaire, and only two items were removed from the questionnaire.14

Multilevel analysis

The mean of all items within a dimension of patient safety culture was calculated, resulting in a dimensional score between 0 and 5 for each of the 11 dimensions. For every dimension of patient safety culture, a three-level multilevel model for continuous responses was fit. The nesting was respondents within units and units within hospitals. For the identification of the units, we did not rely on the reaction of the respondent on the item “unit type,” but we used identification numbers for units and hospitals, printed on the questionnaire.

To control for compositional effects, three individual characteristics were added as covariates: number of years of employment at the unit, length of working week (hours) and occupation (nurse versus other). We corrected for these individual level factors, because we wanted to ensure that the unit differences found in the multilevel analysis were really attributable to differences in patient safety culture and not to differences in group composition (that is, the characteristics of the individual respondents within the units). As a measure of clustering of responses, we calculated the intraclass correlation (ICC) for the unit and hospital level, which is the unit resp. hospital level variance divided by the total variance. This gives a relative measure of the influence of that level on the individual responses. The variances were tested for statistical significance using a one-sided Wald test.16

As an additional test, we fitted a single multivariate model using all the dimension scores. At the respondent and unit level, we fitted a full variance covariance matrix; this was not possible at the hospital level due to the small hospital variance. This is presented in a covariance matrix, showing the interdependency of the dimensions at unit and hospital level. The model was also adjusted for the three above-mentioned covariates. The analyses were done using MLwiN2.

RESULTS

A total of 1889 respondents at 87 units in 19 hospitals completed the questionnaire. The mean response rate (known for 67 units) was 80% (range 25–100%). The number of respondents per unit varied from seven to 53 with a mean of 22 respondents. The mean number of units per hospital was 4.6 (range 2–13). Of all units, 27 were of a surgical specialty, 60 were non-surgical. The majority of the respondents were registered nurses (63%). Most people had worked at their unit for 1–5 years (41%) and worked 20–39 h a week (78%) (table 1). Table 2 gives a description of the dimension scores at the individual, unit and hospital level.

The clustering of responses at both unit and hospital level for each of the 11 dimensions of patient safety culture is shown in table 3. At unit level, all variances were statistical significant. ICCs at unit level ranged from 4.3 (for Openness of communication) to 31.7 (for Adequate staffing). The ICC of 31.7 for Adequate staffing at unit level means that 31.7% of the variance in the outcome Adequate staffing can be allocated to differences between the units. An ICC of 15 is considered quite high.17

At hospital level, ICCs ranged from 0.0 (for Smooth transitions and Supervisor/manager expectations & actions) to 6.2 (for Feedback about & learning from error). The variances of three dimensions were statistical significant. These three dimensions showed substantial clustering of responses at hospital level: Feedback and learning from error (ICC = 6.2), Teamwork across hospital units (ICC = 4.4) and Non-punitive response to error (ICC = 4.5).

All dimensions of patient safety culture showed most of their higher-level variance at unit level.

The interdependency of the dimensions was examined by placing the 11 dimensions of patient safety culture in a multiresponse model (table 4). At individual level, all dimensions were positively interrelated, and there was no large diversity in the magnitude of the correlations. Correlations were small to moderate, ranging from .05 to .44. At unit level, correlations ranged from .02 to .63. At this level, there was more heterogeneity in the size and direction of the correlations. Four relations emerged to be much stronger at unit level than at individual level, namely those between Feedback about & learning from error and Frequency of event reporting (r = 0.63 at the unit level compared with r = 0.36 at the individual level), between Frequency of event reporting and Smooth transitions (r = 0.60 at the unit level compared with r = 0.17 at the individual level), between Overall perceptions of safety and Adequate staffing (r = 0.63 at the unit level compared with r = 0.30 at the individual level) and between Teamwork within units and Openness of communication (r = 0.60 at the unit level compared with r = 0.32 at the individual level). This indicates that, at unit level, the average response to these dimensions—taking the individual adjusters into account—is strongly correlated. So, for example, units that on average score high on Teamwork within units also tend to score high on Openness of communication. Due to smaller numbers and smaller amounts of (co-) variation, it was not possible to calculate correlations between any of the dimensions at hospital level.

DISCUSSION

The results support the claim that the HSOPS measures culture and not just individual attitudes. All dimensions demonstrate significant clustering of responses at the unit level, in accordance with hypothesis 1. Three dimensions also showed clustering at hospital level: Feedback about and learning from error, Teamwork across hospital units and Non-punitive response to error. We hypothesised (hypothesis 2) only one of these dimensions—Teamwork across hospital units—to be a hospital-wide characteristic. The clustering of Feedback about and learning from error and Non-punitive response to error at hospital level may be related to the structure of the incident reporting system in hospitals in The Netherlands at the time of the data collection. At that time, incidents had to be reported to a central hospital committee rather than to the unit where the healthcare provider worked. The reaction of this central committee to each incident report presumably would have been more or less the same, regardless of the unit of the reporter.

In the multiresponse model, there were no highly correlated dimensions of the HSOPS, indicating that each dimension measures a unique aspect of patient safety culture. The strongest correlations were found at unit level; correlations at individual level were smaller and more homogeneous. When controlling for variation at individual and unit level, the effects at hospital level were too small to calculate correlations. This is also an indication that the hospital level is not the most important level with reference to patient safety culture. These findings are in line with some recent studies that have concluded that the unit level is the dominating level for patient safety culture18 and the occurrence of adverse events.19

Our study has some limitations. In the multilevel analyses, the sample size was only 19 at hospital level compared with 87 at unit level. The lower sample size at hospital level, nonetheless, mainly affects the standard errors of the dimension means and, to a lesser extent, the variances between the hospitals. Furthermore, the sample was a convenience sample; we did not obtain a random sample of units and hospitals. However, the sample characteristics demonstrate a good representation of unit types, teaching status and location of Dutch hospitals. We did take random samples of healthcare professionals within each unit, but we did not have any information about the response rates of nearly one-quarter of the units. The response rates of the units for which we did have information, though, were very high, which reduces the likelihood of self-selection bias within units. Finally, data were collected in two rounds: 2005 and 2006. We checked whether this biased the results. There was no systematic effect of the moment of data collection on the results.

The findings of our study have several general implications for strategies aimed at improving the safety culture. Directing all interventions at the hospital level (centralised approach) appears not to be the best method: unit level (decentralised) improvement efforts seem most worthwhile. And since patient safety culture consists of several distinct dimensions, a multifaceted approach is recommended when trying to change the safety culture in a hospital unit.