The culture of safety: results of an organization-wide survey in 15 California hospitals
- 1Center for Health Policy/Center for Primary Care and Outcomes Research, Stanford University, Stanford, CA 94305-6019, USA
- 2Patient Safety Center of Inquiry, VA Palo Alto Health Care System, Palo Alta, CA 94304, USA
- 3National Bureau of Economic Research, Stanford, CA 94305, USA
- Correspondence to: S J Singer, Center for Health Policy, 117 Encina Commons, Stanford, CA 94305-6019, USA;
- Accepted 5 December 2002
Objective: To understand fundamental attitudes towards patient safety culture and ways in which attitudes vary by hospital, job class, and clinical status.
Design: Using a closed ended survey, respondents were questioned on 16 topics important to a culture of safety in health care or other industries plus demographic information. The survey was conducted by US mail (with an option to respond by Internet) over a 6 month period from April 2001 in three mailings.
Setting: 15 hospitals participating in the California Patient Safety Consortium.
Subjects: A sample of 6312 employees generally comprising all the hospital’s attending physicians, all the senior executives (defined as department head or above), and a 10% random sample of all other hospital personnel. The response rate was 47.4% overall, 62% excluding physicians. Where appropriate, responses were weighted to allow an accurate comparison between participating hospitals and job types and to correct for non-response.
Main outcome measures: Frequency of responses suggesting an absence of safety culture (“problematic responses” to survey questions) and the frequency of “neutral” responses which might also imply a lack of safety culture. Responses to each question overall were recorded according to hospital, job class, and clinician status.
Results: The mean overall problematic response was 18% and a further 18% of respondents gave neutral responses. Problematic responses varied widely between participating institutions. Clinicians, especially nurses, gave more problematic responses than non-clinicians, and front line workers gave more than senior managers.
Conclusion: Safety culture may not be as strong as is desirable of a high reliability organization. The culture differed significantly, not only between hospitals, but also by clinical status and job class within individual institutions. The results provide the most complete available information on the attitudes and experiences of workers about safety culture in hospitals and ways in which perceptions of safety culture differ among hospitals and between types of personnel. Further research is needed to confirm these results and to determine how senior managers can successfully transmit their commitment to safety to the clinical workplace.
High reliability organizations (HROs) are those that face high intrinsic hazards yet perform successfully because they treat safety systematically. Classic HROs include commercial aviation, sections of the military, and the nuclear power industry. Safety culture is a major determinant of safety for HROs1,2—for example, safety culture has been formally adopted as a required element of nuclear power safety by the International Atomic Energy Agency.3,4 The theoretical construct of a culture of safety has been applied to many domains. While debate continues over precisely what components are needed for such a culture of safety, several are commonly accepted as being essential (box 1).5,6
Box 1 Components of a culture of safety
Commitment to safety articulated at the highest levels of the organization and translated into shared values, beliefs, and behavioral norms at all levels.
Necessary resources, incentives, and rewards provided by the organization to allow this commitment to occur.
Safety is valued as the primary priority, even at the expense of “production” or “efficiency”; personnel are rewarded for erring on the side of safety even if they turn out to be wrong.
Communication between workers and across organizational levels is frequent and candid.
Unsafe acts are rare despite high levels of production.
There is an openness about errors and problems; they are reported when they do occur.
Organizational learning is valued; the response to a problem focuses on improving system performance rather than on individual blame.
Much of the literature on general patient safety that refers to “safety culture” merely uses it as a synonym for encouraging data collection and reporting,7 reducing blame, getting leadership involved,8 or focusing on systems.9 Activities described in the literature as interventions to address safety culture typically have been applied only in a single institution and did not actually measure safety culture before or after implementation.
Various studies have attempted to measure aspects of culture in healthcare organizations.10,11 Some have tried to correlate cultural types or cultural dimensions with specific care practices or outcomes.12–15 Previous surveys have primarily examined individual work units such as intensive care units rather than entire hospitals. No previous studies have surveyed all hospital personnel including senior managers, physicians, and other employees, nor have published studies included a diverse set of institutions.* In addition, some previous studies have asked respondents to classify their institution into predefined cultural types, but these have not assessed actual respondent attitudes relative to safety culture.16,17
The Center for Health Policy and Center for Primary Care and Outcomes Research (CHP/PCOR) at Stanford University and the Patient Safety Center of Inquiry (PSCI) at the VA Palo Alto Health Care System conducted a safety culture survey with two principal objectives:
(1) to measure attitudes towards patient safety and organizational culture in 15 hospitals participating in the California Patient Safety Consortium (PSC). Hospitals in the PSC were selected non-randomly through recommendations by investigators and consultants for their initiative in promoting patient safety within their own institution, their diversity in size, ownership type, hospital system affiliation, and their geographical location (mostly in Northern California; see table 1);
(2) to determine how the “culture of safety” varied among the hospitals and between different types of healthcare personnel in order to identify opportunities for improvement in participating institutions and to establish a baseline for assessing future improvement efforts. We focused on responses to survey questions that represent attitudes and experiences indicative of the absence or antithesis of the elements of a culture of safety articulated by high reliability organization theory. Such responses were termed “problematic responses”. We hypothesized that, if hospitals truly are HROs, the level of problematic responses would be low (10% or less). We also hypothesized that responses of senior managers would differ from those of front line workers, as would those of clinicians versus non-clinicians.
The survey instrument, initially constructed by PSCI, was adapted with permission from five existing surveys (OR management attitudes questionnaire, anesthesia work environment survey, naval command assessment tool, risk management questionnaire, and safety orientation in medical facilities16,18–22). A review of these instruments produced partially overlapping questions covering 16 topics plus demographic information (see box 2). Many of these topics have been identified as important to a culture of safety.10,18,23–25
Box 2 Stanford/PSCI culture survey: survey topics and demographic information
Reporting of mistakes
Rewards and punishment for reporting
Feelings of blame and shame
Risk perception—differences in the perception of risks between senior management and practitioners
Process auditing—institutional auditing of patient safety incidents
Time and resources—availability of various resources to treat patients safely
Mitigating decisions—level and quality of decision makers
Organizational structure—general safety climate and communication infrastructure
Fatigue and stress
Quality of hospital operations
Redundancy—ability of experienced personnel to improve patient safety
Rules and procedures within the hospital
Respondent’s position within the hospital, management level, age, and sex
The preliminary survey instrument was tested extensively in pilot studies at VA facilities before implementation at the Consortium hospitals. After modification based on feedback about specific questions in the pilot studies, a revised instrument (82 questions plus demographic information) was distributed to Consortium hospitals in two mailshots. For the third and final mailshot an abridged version (30 questions plus demographic information) was created by eliminating overlapping questions while retaining at least one (and usually two) questions for each of the 16 topics. All questions were close ended. Three scales were used: a 5-point Likert scale; a yes/no/uncertain scale; and a 5-point frequency scale (always/frequently/sometimes/rarely/never). Unless otherwise specified, results reported here pertain only to the 30 questions included in all three mailshots.
Approval to conduct this survey was granted in advance by the relevant institutional review boards of all of the participating institutions.
At each participating hospital the target sample was 100% of the hospital’s attending physicians, 100% of senior executives (defined as department head or above), and a 10% random sample of all other employees. For logistical reasons two hospitals surveyed less than 100% of physicians (one large hospital sampled 250 of their top admitting physicians, 25% of all physicians, and the other hospital randomly sampled 20% of their physicians). In a third hospital too few responses were received from physicians to analyse them as a separate category. The initial mailing list contained 6906 names; 347 duplicates were removed and 227 were returned as undeliverable (employee retired, was a temporary worker, or no longer employed), leaving a total of 6312 eligible individuals in the 15 hospitals.
Administration of survey
Between April and June 2001 the hospitals distributed survey packs to recipients via inter-office or US mail in three mailshots. There was some variability in starting dates between institutions for logistical reasons, and successive mailshots were separated by approximately 4 weeks. The packs included a cover letter co-signed by an investigator and a senior executive from the applicable hospital, a paper survey instrument, a business reply return envelope, instructions for completing the survey on the Internet if desired, and a separate questionnaire completion notification (QCN) postcard (first two mailshots only). A secure web version of the survey was offered for the first two mailshots. The QCN postcard allowed us to track non-responders to each of the first two mailshots without compromising the anonymity of the survey responses.
Weighting of data
Two weights were applied multiplicatively to the raw data to allow accurate comparison between participating hospitals and job types and to correct in part for subject non-response.
To correct for the different sampling strategy for executives and physicians (100% sampling) versus other employees (10% sampling), we weighted the responses of other employees by a factor of 10 relative to executives and physicians. For hospitals that surveyed fewer than 100% of physicians or senior managers the sampling weights were adjusted accordingly. This weighting was applied whenever responses from different job types were aggregated together.
To correct for the different rate of non-response to the survey from employees of different job types, the primary data set was created by weighting the respondents for each of three job types by the inverse of the non-response rate for that job type from all mailshots.26,27 For example, if the overall response rate across all mailshots of management, physicians, and other employees was 66%, 33%, and 50%, the relevant respondents would be weighted by factors of approximately 1.5, 3, and 2 to account for non-responders. We also created alternative data sets that accounted for the non-responders by weighting the mailshots separately. Responses to the first mailshots were significantly more likely to suggest the absence of a culture of safety—that is, they were more “problematic” about safety culture than were those to the third mailshot. Weighting respondents only from the first and second mailshots to account for those who did not respond to any mailing yielded a data set that gives a relatively pessimistic estimate of safety culture by the non-responders. Weighting only the data from the third mailshot to account for non-responders created a more optimistic estimate of non-responder attitudes.
Analysis of data
To allow aggregation of the different survey questions and comparison between hospitals and job types, the “problematic responses” to each question were compared. The answer choice that constituted a “problematic” response varied by question, depending on how the question was phrased. For example, in response to “I will suffer negative consequences if I report a patient safety problem”, to agree or strongly agree was considered problematic. In contrast, in response to “I am rewarded for taking quick action to identify a serious mistake”, to disagree or strongly disagree was problematic. We also examined the frequency of neutral responses, as these might also imply a lack of safety culture (L Kearney, personal communication). “Neutral” responses were neutral on questions using a 5-point Likert scale, uncertain on questions offering yes, uncertain or no responses, and sometimes on questions using a 5-point frequency scale.
Principal factor analysis was conducted on the responses to the 82 question survey implemented in the first two mailshots (with a varimax rotation to maximize the loadings on each factor). This yielded five factors which accounted for 80% of the systematic variation across questions. Based on the questions that loaded most heavily to each factor, we labelled these factors as “organization”, “department”, “production”, “reporting/seeking help”, and shame/self-awareness”, respectively.
We also analysed responses to questions by respondent hospital, job class, and clinician status. Although this report contains primarily a descriptive analysis, we also conducted multivariate regression models with interaction terms and hospital fixed effects. We comment briefly on these results where they add to information provided by the descriptive analysis. In addition, by comparing responses in our primary data set with those in the pessimistic and optimistic data sets (imputing different patterns of response to the non-responders), we conducted a form of sensitivity analysis on the probable impact of non-response bias.
Because the tables for the multivariate regressions and alternative non-response weighted data sets are extensive, they are not included but are available electronically from the authors.
Meeting the objectives of the study
Measuring the problematic response to survey questions enabled us to meet our principal objectives—to measure attitudes towards safety culture and to determine how safety culture varies among hospitals and between types of personnel. Our findings establish a baseline for future benchmarking and identify opportunities for improvement in participating hospitals.
A total of 2989 surveys were returned over a 6 month period giving an overall response rate of 47.4%. The response rate excluding physicians was 62%, and the response rate for physicians was 33%. The overall response rate varied by hospital, ranging from 38% to 71%. The response rate also varied by mailshot. Of the 6312 subjects, 25% responded to the first mailshot, 9% (12% of the remaining subjects) responded to the second mailshot, and 13% (20% of the remaining subjects) responded to the third mailshot. The increase in response to the third mailshot may be attributed to the shorter survey instrument used. Respondents returned nearly all surveys by US mail; only 1% of respondents used the Internet.
Overall, there were slightly more women among the respondents (54%, table 2). The sex of the respondents varied with job class, reflecting the different sex profiles of the medical and nursing professions; 29% of physicians, 51% of senior managers, and over 70% of other employees were women. 64% of the sample were aged between 40 and 60 years. 65% were physicians, nurses, pharmacists, or other core clinical staff; 31% were non-clinicians, and the remainder (4%) were other clinicians such as nursing assistants or therapists.
Attitudes toward patient safety and organizational culture
The average problematic response for the 30 questions common to all three mailshots was 18% (table 3). Problematic responses to individual questions ranged from 4% to 52%. For 21 of the 30 survey questions problematic responses were more than 10%; for eight questions problematic responses were more than 25%. Problematic responses were higher for questions that were phrased as hypothetical or impersonal, and lower to questions that were personal or time delimited.
When neutral responses were included, problematic + neutral responses averaged 36.5% across all questions (range 8–70% for individual questions). Not surprisingly, personal questions relating to quality of care yielded more neutral responses than did those concerning coworkers.
Many respondents indicated a lack of rewards and fear of punishment for identifying and making mistakes. Approximately 33% of respondents said they were not rewarded for taking quick action to identify a serious mistake, and 28% believed they would be disciplined if a mistake they made was discovered. Respondents also felt that pressure impacted on patient care, noting a small but appreciable incidence of unsafe acts. Almost 52% believed that loss of experienced personnel had negatively affected their ability to provide high quality patient care. More than 39% reported that they had witnessed a coworker do something that appeared (in their opinion) to be unsafe, and 8% admitted that in the last year they had done something that was not safe for the patient.
Variation between hospitals
There was wide variation between participating hospitals, despite the presumed attention to patient safety by all of them. The average overall problematic response across all questions varied in individual hospitals from 13% to 22%, a 9% spread. For individual questions the range in problematic response varied from 6% to 38% between institutions. Problematic responses between hospitals varied most in answer to “Individuals in my department are willing to report behavior which is unsafe for patient care” and “Asking for help is a sign of incompetence”. While the hospital with the best survey results had only 13 questions with more than 10% problematic responses, the hospital with the worst results had 25 questions above this threshold (data available from authors).
Variation between types of hospital personnel
When the problematic response rates were compared by job class (senior managers, non-senior managers) and clinician status (clinician, non-clinician), clinicians in general were found to be more likely to provide problematic responses than non-clinicians (table 4). In addition, senior managers in general were less likely to give problematic responses than non-senior managers. Senior managers reported up to 15% fewer problematic responses on individual questions. The only question to which senior managers responded with more problematic responses than front line workers was “Telling others about my mistakes is embarrassing”.
Multivariate regression analysis confirmed these conclusions (p<0.01) and further suggested that responses by clinician senior managers were more similar to responses by other clinicians than to those by non-clinician senior managers (p<0.01)—that is, clinicians were generally more negative than non-clinicians, regardless of management status. Among clinicians, nurses were the most negative (p<0.01) and almost always responded significantly more negatively than non-clinicians (25 out of 30 questions). Similarly, sensitivity analysis of the effects of non-response bias (using the pessimistic and optimistic estimates of non-responders) showed that these findings were unlikely to be due to selection bias.
Because these data were acquired using systematic sampling of a full cross section of employees in a diverse set of institutions, the results provide the most complete available information on the attitudes and experiences of workers about safety culture in hospitals. While most responses were consistent with a culture of safety, a substantial minority of respondents gave a problematic response (box 3). Across all questions and institutions the aggregate problematic response was 18% (37% including neutral responses). For 70% of the questions in our survey the aggregate problematic response was above 10%, and for 27% it was above 25%. Respondents also reported a small but appreciable occurrence of unsafe acts.
Box 3 Key findings from the patient safety survey
The majority of participants in the survey responded in ways which indicated a culture of safety.
A substantial minority of respondents gave a problematic response.
Respondents reported a small but appreciable incidence of unsafe acts.
Clinicians gave more problematic responses than non-clinicians; among clinicians, nurses were most pessimistic.
Senior managers gave fewer problematic responses than front line workers.
One key finding of this study is that there was a definite discrepancy between the attitudes and experiences of senior managers (particularly non-clinicians) and those of non-managers. Non-clinician senior managers answered more often in ways consistent with a culture of safety than did personnel who actually take care of patients. Clinicians, particularly nurses, gave more problematic responses than non-clinicians, regardless of management status. These findings have implications for managerial strategies in hospitals (see below).
Relationship with previous research and patient safety initiatives
Compared with the findings of Sexton and colleagues,10 the levels of problematic response reported in this survey are no more problematic than responses provided by operating room and intensive care clinicians to questions dealing specifically with fatigue, stress, hierarchy, and teamwork. However, those studying HROs believe that a high uniformity of safety attitudes and experiences is necessary to realize a working safety culture. They raise concerns when problematic attitudes exceed about 10% of a cohort (A Ciavarelli and K Roberts, personal communications). By this standard the number of problematic responses in participating hospitals was still worrisome, even in the hospital with the best survey results (overall problematic response of 13%).
The aggregate subject response rate in this survey (47%) was not as high as desired, largely because of a low response among physicians. However, we predicted a lower response rate for physicians which is why we chose to sample nearly 100% of them. The actual number of physician respondents was over 1000. In addition, the responses of physicians were similar to those of other clinicians who had a much higher response rate. We believe the overall response rate was satisfactory for a complex survey administered to extremely busy individuals. We were able to make some estimates of possible non-response bias by incorporating the response wave into the analysis. This suggested that the overall results would probably not be affected strongly by non-response bias.
Our results may not be representative of hospitals generally as the hospitals surveyed in this study were members of a consortium of institutions with a specific interest in advancing their performance in patient safety. We presume that these are probably leading institutions in safety rather than laggards, although it is possible that some participated because they were aware of serious deficiencies and wished to correct them. The data from the survey are therefore more likely to reflect a best case rather than a worst case scenario. The small number of hospitals surveyed to date does not yet permit us to consider statistically significant differences between types of hospitals.
The discordance found between non-clinician senior managers and front line workers may reflect a tendency for front line workers and middle managers to gloss over patient care problems in briefings to senior managers.28 This could make it hard for non-clinician executives to understand the true state of their organization, to determine changes needed, and to assess their attempts to create and maintain a culture of safety. In addition, these results could imply inadequate communication to the front line workers of the commitment of the management to patient safety. If further studies confirm these findings, reasonable strategies for improving safety culture might involve creating better linkages between executives and those directly concerned with patient care. Our findings also suggest that, while short term efforts to work on specific interventions (such as computerized physician order entry implementation) are laudable, they may be insufficient if the managerial structures and cultures of institutions do not sufficiently favor patient safety. Long term progress may need to include interventions specifically aimed at improving safety culture and breaking down barriers between managers and front line workers.
Further studies are required to determine more precisely the magnitude of the problematic response that indicates an unsafe culture. Using our instrument in a much larger set of hospitals over a larger geographical area could confirm the results in a broader range of personnel, and would allow assessment of the relationship between safety culture and hospital characteristics. Longitudinal studies could allow measurement of the impact of interventions designed to address safety culture. Comparing organizations in a variety of industrial domains and correlating the results with the actual safety performance of these organizations would provide important validation of the results. Such studies would provide a solid basis upon which to recommend significant policy changes and long term reform strategies to address safety culture.
Finally, since we cannot fully exclude non-response biases in our study, future studies may need to use intensive surveying of a subsample of non-responders (especially physicians) to test whether those who respond are systematically different from those who do not.
The authors wish to thank the hospitals that participated in the survey and especially the PSC coordinators within each hospital who provided essential assistance in conducting this survey. We also thank Julie Harper (VA Palo Alto) and Jennie Bowen (Stanford) for their research assistance.
↵* The Veterans Health Administration surveyed personnel from VA hospitals nationally using two instruments including a version of the VA Palo Alto Patient Safety Center of Inquiry culture survey used in this study. However, non-uniformity in the administration of the survey rendered the results difficult to interpret.
This project is funded by a grant from the Agency for Health Care Research and Quality (U18 HS011114) and also by the Patient Safety Centers of Inquiry program of the Veterans Health Administration.
Some of the data appearing in this paper were presented at “Annenberg IV: Patient Safety: Let’s Get Practical,” Indianapolis, IN, 22–24 April 2002; “Making the Health Care System Safer: AHRQ’s First Patient Safety Research Conference,” Alexandria, VA, 7–8 February 2002; and “Keys to Patient Safety: California Association of Healthcare Quality Conference,” Stanford, CA, 5 October 2001.