In response to growing concern about patient safety, the Department of Health in the UK¹ and the Institute of Medicine in the USA² advised that healthcare organisations should consider adopting the safety management techniques used in other industries. The UK industrial safety regulator, the Health and Safety Executive, recommends that organisations operating in high risk industries should regularly assess their safety culture.³ Safety culture is “the product of individual and group values, attitudes, perceptions, competencies and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation’s safety management” (page ).⁴ This is usually measured in industry by workforce questionnaire surveys to assess what is called the “safety climate”.⁵ Safety climate can be regarded as the surface features of the underlying safety culture.⁶ It assesses workforce perceptions of procedures and behaviours in their work environment that indicate the priority given to safety relative to other organisational goals.^7–9 As organisations are inherently hierarchical in structure, there are multiple levels at which safety climate can be investigated—for example, individuals, work groups, departments, organizations.¹⁰ Safety climate data are generally collected at the individual level, then aggregated to a higher level. The degree of homogeneity of workers’ perceptions, as a measure of climate strength, can also be considered.^11,12

A number of different instruments are used to measure safety climate in industry.^6,13 The resulting data offer managers an additional perspective on the state of their safety management systems and can also be used for benchmarking purposes and trends analysis.¹⁴ It has been argued^15,16 that the essential dimension is management commitment to safety: while this is probably fundamental, industrial researchers do measure other aspects. The most common^6,13 are shown in box 1. In industry, workforce perceptions of safety climate have been linked to safety outcomes such as workforce injuries,^17–21 and to safety processes such as workers’ behaviours.²²

Safety climate surveys are now being increasingly used in healthcare organisations²³ and several instruments have been developed. This paper reviews quantitative studies designed to investigate safety climate in health care, with particular attention devoted to the questionnaires. It provides a complementary analysis to a recent review of survey instruments,²⁴ with some overlap in the studies examined.

METHODS

Four databases were searched: MEDLINE, PsychINFO, EBSCO, and Web of Science using the search terms “health care workers”, “hospital safety”, “patient safety”, “safety climate”, and “safety culture”. Relevant papers were retrieved and papers were also retrieved from patient safety conferences. A total of 29 studies were initially identified. The criteria for inclusion for detailed scrutiny were: (1) use of a questionnaire for individual response designed to measure safety climate or safety culture in a healthcare setting; (2) details provided of the measuring instrument; (3) tested on a sample of over 50 respondents; and (4) report published in English. From the 29 papers retrieved, 12 studies were identified as suitable for review.^25–36 Studies reporting different components of the same data set^37–41 were only included once, and those that examined general organisational culture or climate variables (such as work pressure or role ambiguity) in relation to safety in health care^42–44 were not included. The analysis extracted information on the survey location and sample, safety climate measure, safety outcome variables, and the main findings. The specific psychometric properties^45,46 considered (box 2) were the content validity, criterion validity, as well as the internal factor structure of the instrument.

The safety climate factors/dimensions given in each study (table 1) were independently categorised (with acceptable inter-rater agreement) by two industrial psychologists (CB, SY) into 10 themes corresponding to distinct aspects of safety management. This was carried out by examining the content of the items loading on each subscale/factor where these were available. As most of the studies had based their definitions of safety climate on the literature from industry or had adapted industrial instruments, the themes were labelled in a similar fashion to those most commonly measured in industry (box 1).

RESULTS

The 12 studies are described in table 1 in terms of the survey sample, instrument details (with any psychometric data), identified safety climate factors, outcome measures, and results. Most of the studies were from the USA and most sampled medical staff in different occupations (response rates 26–91%). The first nine studies used different safety climate measures while the other three used different versions of the Operating Room Management Attitudes Questionnaire (ORMAQ).⁵⁴ The ORMAQ was not originally designated as a general healthcare safety climate measure but recently it has been used for this purpose, so these studies were included for review. (Studies with the ORMAQ that did not specifically claim to be measuring organisational safety climate^55,56 were excluded.)

Drawing from the information presented in table 1, the questionnaires were examined with respect to content validity, factor structure and internal reliability, and criterion related validity. The level of analysis used in the study is briefly considered below.

Content validity

Nine of the studies set out to measure safety climate, the remainder used the term safety culture, and one³⁰ used both terms. Four did not define either term. Definitions of safety climate were usually a version of shared perceptions of safety.¹⁰ A theoretical basis or model to outline proposed causal influences between safety climate and the safety outcome measures was rarely specified. One exception³² stated that their survey items were based on “elements of a culture of safety articulated by high reliability theory” (page ). They listed seven components derived from this theory⁵⁸ but it is not clear how these relate to their questionnaire items and the five extracted factors did not provide confirmation. A circular model was given in another study²⁶ but it did not articulate any explanatory mechanism between safety climate and safety behaviours.

The 73 safety climate dimensions (table 1) were categorised into 10 safety management themes (box 1). Not surprisingly, given the origins of several of the measures, there was considerable overlap with the features measured in industry. Management commitment to safety emerged as the most frequently measured safety climate dimension in health care with nine studies including this. Three included supervisor commitment to safety. Safety systems—for example, personal protective equipment and safety training—were included in seven studies. Unlike the industrial sector, general attitudes to risk were not specifically addressed.

Work pressure is an important safety climate feature in industry and three healthcare instruments included a job demands/workload dimension. Unlike the industrial sector, competence did not emerge as a separate dimension, although two studies included measures about training and one included knowledge about universal precautions. Lastly, and in contrast to industry, compliance with procedures/rules did not emerge as a separate dimension in the healthcare measures, although two studies measured whether unsafe work practices were corrected by supervisors and workmates.

From the comparison it seems that at least three “core” dimensions from industry are being measured as components of safety climate in health care—management/supervisory commitment to safety, safety system, and work pressure. Sorra and Nieva³⁴ included two of these dimensions but did not assess perceptions of the safety system. Pronovost et al³⁶ also included two of these dimensions but their measure of the safety system was quite limited in scope as their safety climate scale only contained 10 items. Most of the other studies included one or two of these “core” dimensions each. This lends some weight to the argument for a set of universal or core variables that underpin safety climate across work sectors, although these probably need to be complemented with sets of sector specific factors.^8,11

Factor structure and internal reliability

For this review, studies with a sample size of less than 300 and factors consisting of three items or less were regarded with caution (box 2). Two studies conceptualised safety climate as a unidimensional construct but did not report a factor analysis to confirm this. Only six studies reported the results of a factor analysis. DeJoy et al²⁵ conducted separate exploratory factor analyses on their 35-item measure of safety climate for each of the occupational groups studied (nurses, physicians, technicians). Each analysis yielded eight similar factors based on the same 23 items, as shown in table 1 (α coefficients ranged from 0.61 to 0.83 apart from general work organisation which was 0.39). However, more than half of these factors (feedback, knowledge and information, perceived risk, response efficacy, work organisation) were based on three items or less, which is usually regarded below minimal. Gershon et al²⁶ conducted an exploratory factor analysis on their 46-item safety climate scale, yielding six factors (based on 20 items) that did not correspond to their nine hypothesised safety climate dimensions. An internal reliability analysis of the factor scales yielded acceptable α coefficients. Singer et al³² found a five-factor solution that did not match their original thematic groupings. In two studies the data sets were rather small for the factor analyses that were conducted.^29,30

Only one study provided a comprehensive report of scale development. For their Hospital Survey on Patient Safety, Sorra and Nieva³³ conducted an exploratory factor analysis to explore the dimensionality of their survey data. The results of the exploratory factor analysis revealed the existence of 14 distinct factors. A subsequent confirmatory factor analysis was conducted and the final confirmatory factor model contained 42 items in 12 factors (two factors which measured outcomes and 10 which measured safety climate). This model fitted the data well with good α coefficients.

Three studies used versions of the ORMAQ.^34–36 Despite sample sizes of more than 300, none of them reported a factor structure or internal reliabilities for the hypothesised dimensions. Itoh (personal communication, 2003) indicated that the results of an attempted factor analysis were not interpretable; a similar result has been reported elsewhere.⁵⁶

Criterion validity

Criterion or outcome measures of safety in health care could include worker behaviours, worker injuries, patient injuries, or other organisational outcomes (such as litigation costs). As shown in table 1, four studies had no outcome measures. Self-reported worker rule compliance or event reporting behaviours were recorded in five studies, with two including independent measures of occupational injuries.^28,29 Only one study had an independent measure of injuries or annoyances to patients,³⁴ although this was broadly conceptualised and included events such as losing artificial teeth. Other studies used the term “patient safety culture”^32,35 but included no patient outcome measures.

The three studies using independent outcome measures reported significant associations between climate scores and outcomes. Two^28,30 found evidence that positive perceptions of organisational safety in healthcare settings were related to fewer worker injuries. Itoh³⁴ measured the relationship between rates of nurses’ reports of patient incidents from one of the participating hospitals from the preceding year and aggregated scores on the questionnaire. There was no correlation between questionnaire responses on incident reporting and rates of adverse events to patients. A significant negative correlation was reported between questionnaire scores on “recognition of inevitability of human error” and rates of incident reporting. This represents one of the few attempts to include an independent measure of worker safety behaviour but, because of the breadth of the measure and the small sample of units, these results would require to be interpreted with a degree of caution. Correlations of climate scores with self-reported safety behaviours generally showed positive relationships.^25,27,33

Some care needs to be taken with the level of analysis for measuring and testing predictors against outcomes (that is, individual worker, team, department, hospital, NHS trust).^59–61 Only one study³⁴ attempted to examine the relationship between safety climate of work units and their safety outcomes. Most aggregated questionnaire responses from the entire sample which often included more than one hospital.^25,28,32,33 While Gershon et al²⁶ aggregated responses to the organisational level, their sample only included hospital employees who were at risk of exposure to blood and body fluids. Neal et al²⁷ and Pronovost et al³⁵ aggregated responses from representative samples of hospital workers to the organisational level. Although it can be argued that these studies produced more meaningful safety climate data, they did not examine the relationship with organisational safety outcomes such as worker or patient injury rates.

DISCUSSION

The UK National Audit Office⁶⁵ has recently reported on the state of patient safety in NHS trusts. While offering an encouraging prognosis, this is far from a clean bill of health. The report states that “The safety culture within trusts is improving … However, trusts are still predominantly reactive in response to patient safety issues and parts of some organisations still operate a blame culture” (page ). Measuring safety climate in health care helps to diagnose the underlying safety culture of an organisation or work unit. The prevailing culture influences safety behaviours and outcomes for both healthcare workers and patients. Safety climate questionnaires need to achieve as high a standard of measurement as possible so that healthcare managers can use the resulting data to design effective safety management systems and interventions.

We have reviewed the psychometric properties of instruments used to measure safety climate in 12 studies based in healthcare settings. None of these had achieved full scale testing and it is recognised that some instruments were at an early stage of development. The Hospital Survey on Patient Safety³³ met more of the specified psychometric criteria due to more systematic testing of internal structure than the other instruments reviewed. Some of the scales of this instrument—such as organisational learning/continuous improvement and teamwork (within, and across hospital units)—should not perhaps be considered part of the safety climate¹¹ unless their relationship with safety outcomes can be confirmed. This study only had a 29% response rate which was rather low compared with the other studies and may signal issues of usability or weaknesses in their survey method. Medical staff have limited time to complete and return questionnaires, so instruments for health care may need to be parsimonious and made available electronically as well as on paper to maximise response rates.

Several of the instruments had been developed from measures used in other industries (aviation, oil, nuclear). Considerable care needs to be taken when adapting measures from these very proceduralised high risk industries. Not only is the nature of the work very different, but the organisations have well defined hierarchical management structures with clear reporting relationships. Leadership issues are much more problematic to measure in health care as the managerial reporting relationships are subject to different interpretation by each professional group, thus introducing a degree of ambiguity. This is particularly true for doctors.⁶⁶ Moreover, the safety climate studies in industry all focus on worker injury rather than product (cf patient) damage. Determining reliable outcome measures for these healthcare studies appears to be challenging; sometimes the focus is on workers’ behaviours, which might be regarded as safety process measures, and in other cases some kind of adverse event is used. As more patient safety indicator and outcome measures are being introduced, these should permit stronger data based on work unit and organisational performance to give appropriate higher level criterion metrics.

The data sets were drawn from different levels of organisational analysis and, as mentioned earlier, nested data of this type need to be analysed with some care—for example, by using multilevel modelling statistical techniques. Aggregating safety climate data across hospitals and, indeed, across healthcare systems in different countries⁶⁴ is not entirely advisable unless the questionnaire is measuring sector rather than organisational features. Zohar¹¹ has argued that safety climate can be meaningfully construed only at the group and organisational levels so as to reflect a particular supervisor’s and senior management’s influence on safety, respectively.

This is a preliminary review and it should be acknowledged that many of these research teams are now engaged in larger scale questionnaire studies. In future, meta-analysis based on effect sizes will be needed to compare their results and to determine the validity and generalisability of the climate measures. In general, these studies have begun to confirm that safety climate scores can be associated with healthcare workers’ safety behaviours or workers’ injuries, replicating earlier findings from industry, although few independent measures were used. Very few of the reviewed studies considered the mechanisms that mediate the relationship between safety climate and safety outcomes (that is, worker injury or patient harm). In the wider literature on safety climate there are now models that attempt to explain the psychological mechanisms linking safety climate and worker behaviour.^9,11 In these models the relationships between safety climate, safety behaviour, and safety outcomes are focused on individual worker injury. In the healthcare sector there is an additional need to establish whether a different set of antecedents influences processes (worker behaviours) that affect patient safety outcomes as opposed to worker injury. In other words, are there different motivating factors that determine the safety behaviour of healthcare workers in relation to preventing personal harm compared with harm to a patient? The Institute of Medicine report¹ “To Err is Human” stated that “workers’ safety is often linked with patient safety. If workers are safer in their jobs, patients will be safer also” (page ). In fact there is little evidence to support this claim, although emerging evidence is encouraging.⁶⁷ So future safety climate research in health care should elaborate and test models that attempt to explicate the mechanisms influencing not only patient safety but worker safety as well.

Finally, while questionnaires offer an efficient and anonymous method of collecting safety climate data, researchers need to consider alternative techniques for sensing organisational culture. Ethnographic approaches based on observation and interviewing⁶⁸ can be expensive but they can provide valuable qualitative data to test the validity of the survey methods.