“The journey of a thousand miles begins with one step.” Latsu

Safety culture is increasingly recognized as an important strategy—and perhaps a necessary precursor—to improving the widespread deficits in patient safety. The Joint Commission for Accreditation of Healthcare Organizations (JCAHO) included an annual assessment of safety culture in its 2007 patient safety goals. The Institute of Medicine (IOM) report “To Err is Human” spurred healthcare organizations to implement initiatives that improve patient safety.¹

Despite this, culture is defined and measured in various ways. Definitions of culture commonly refer to values, attitudes, norms, beliefs, practices, policies, and behaviors of personnel. In essence, culture is “the way we do things around here”, whereby the word “here” refers not to the hospital, but rather to a particular work unit. In a safe culture employees are guided by an organization-wide commitment to safety in which each member upholds their own safety norms and those of their co-workers. A number of tools are available to measure safety culture, but each instrument has unique domains of culture, limited validity and reliability data, and average response rates that vary from poor (29%) to excellent (83%). The science of measuring safety culture is evolving, even as the demand for rigorous cultural assessment intensifies.

MEASUREMENT OF SAFETY CULTURE

Many organizations have embarked on efforts to measure safety culture. It is not uncommon for senior leaders in these hospitals to use culture survey scores as a system level measure of patient safety to hold managers accountable, often with the use of bonuses. Although these efforts are laudable, the enthusiasm for measuring culture may be outpacing the science. Due perhaps to the nascent nature of cultural assessment in health care, culture researchers lack consensus and clarity about domains important in a culture of safety; how to score and present improvements in culture over time; the relationship between culture and clinical and operational outcomes; and ultimately, how to package a tool kit to measure, score, and improve culture.

An important and perhaps glaring gap in our knowledge of cultural assessment is understanding the sources of variation in culture—that is, we do not understand whether staff characteristics, the patient care area, the department (where applicable), or the hospital explain variation in culture. We must understand these sources of variation in order to target who to measure, how to score, where to focus efforts to improve culture, and to hold accountable for improving culture.² Failure to understand these important issues can cause managers to make incorrect inferences regarding scores on cultural assessments, and potentially cause additional harm by either rewarding the wrong behaviors or diverting scarce resources away from important efforts.

A first step in this effort to measure safety culture is to ensure that the survey instruments are valid and reliable—that is, that they measure what they intend to measure and produce similar results upon repeat measurement. The paper by Kho and colleagues³ published in this issue of QSHC does much to advance our science of measuring culture. They adapted and administered the Safety Climate Survey to measure safety culture in four Canadian University affiliated intensive care units (ICUs). Using a novel technique, they achieved a 75% response rate and evaluated the validity and reliability of three different scoring methodologies. They measured internal consistency to estimate validity, and to determine reliability they measured test-retest reliability. They found that the Safety Climate Survey overall (22 items) and the 13-item scale had construct validity and sufficient reliability while the 7-item scale lacked construct validity and, as such, was not considered further. The high score on internal consistency suggests that the questions on the full and 13-item subset measure a single construct—safety culture. The authors are to be applauded for their rigorous evaluation of construct validity and reliability of these scoring methodologies. They add important new knowledge on how to measure culture.

ORIGIN OF SAFETY CLIMATE SURVEY

It is important to recognize the origin of the Safety Climate Survey used by Kho et al. In 2002 we extracted a subset of items related to safety climate from the larger Safety Attitudes Questionnaire.⁴ This subset of safety climate items did not elicit attitudes along any dimensions such as teamwork climate or perceptions of management. We provided a copy of this survey with instructions and comparison data to the Institute for Healthcare Improvement for posting on their new website (www.qualityhealthcare.org).

Our own analyses of the Safety Climate Survey scoring methodologies showed test-retest reliability of 0.85–0.90 and Cronbach α values of 0.75–0.88.⁵ These psychometrics are sound, yet they tell us nothing about which domains of culture are most appropriate. In our experience with the multidimensional Safety Attitudes Questionnaire, we have 30 items that measure six domains: safety climate, teamwork climate, perceptions of management, stress recognition, job satisfaction, and working conditions. We are only beginning to appreciate what we gain from multidimensional cultural assessments. Clearly, the additional dimensions are informative and allow feedback to hospitals and work units to consider a number of their cultural strengths and weaknesses.

When we feed back the results of these surveys, which we have now done at over 500 hospitals, caregivers and managers generally find the multidimensional cultural feedback and comparison data highly informative. We have shown that focused interventions can improve the safety and teamwork climate in a given work unit, yet these often improve at the expense of another cultural dimension—namely, stress recognition. For example, as work units evolve to be highly efficient with greater trust, collaboration, and openness, caregivers sometimes develop a sense of personal invulnerability as a by-product of working on a stellar unit. Unfortunately, this reduction in acknowledging stressors is a dark path that leads otherwise outstanding work units to host seemingly unexplainable sentinel events. In other words, we need the more diagnostic multidimensional cultural assessment tools to have the ability to track units on a variety of strengths and weaknesses. The Safety Attitudes Questionnaire has good construct validity and internal consistency, yet there is a tremendous amount of work still to be done on criterion validity for each domain which would show how dimensions of culture link to clinical and operational outcomes. This is an active area of research.

RECOMMENDATIONS FOR THE FUTURE

What would we recommend for healthcare organizations interested in measuring safety culture? Our recommendations are informed by our prior mistakes and continued research. We would use the full Safety Attitudes Questionnaire (rather than just safety climate) and measure the entire hospital annually; this has already been done at the Johns Hopkins Hospital. We found that when you measure and feedback data in one work unit, other units quickly desire their own cultural assessment as well. The Safety Attitudes Questionnaire is the most widely used cultural assessment tool in health care to date. In the past 12 months we have assessed culture in over 100 hospitals with an average hospital-wide response rate of over 80%. Representativeness is critical as it makes the data easy to interpret and difficult to ignore. This is particularly true in pre-post or longitudinal cultural assessments where high response rates are essential to interpreting data over time. When response rates fall below 60%, the data represent opinions rather than culture and the results should be used with caution.

In addition, the measurement of culture should include a presentation of results to staff as well as senior management, followed by a focused intervention to improve culture. There is limited evidence regarding interventions that improve culture. To our knowledge, the Comprehensive Unit-based Safety Program (CUSP) is the only published intervention that has been shown to improve culture.⁶ Even with a valid measure of culture, if culture is not responsive to interventions there is no point in measuring it. Although far from perfect, the CUSP provides a practical framework for improving patient safety (culturally, clinically, and operationally) throughout an entire organization by focusing on individual work units and respecting the wisdom of their frontline providers.

Why focus on the unit level? Culture is local. Intervening in culture requires focusing at the local work unit level. Figure 1 shows the safety climate for 100 hospitals where the hospital level safety climate ranges from 40% to about 80% positive. We pulled out hospital X from fig 1 to show the typical variability within a hospital across the work units. Figure 2 shows that, within hospital X, positive safety climate scores range from 0% to 100% among work units. With few exceptions, we find more variability between work units within a hospital than we do between hospitals. For this empirical reason, it is critical to assess culture across all work units in an institution.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

 Safety climate across 100 hospitals.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

 Safety climate across 49 work units in one hospital.

The research by Kho et al³ provides us with new confidence in the ability to move towards measuring safety culture with methodological rigor. Future research is needed to evaluate whether measuring additional domains such as teamwork climate, perceptions of management, or stress recognition is useful. Let us hope that this well written and rigorously conducted paper is an early step in a long journey towards understanding safety culture and ultimately improving patient safety.