Article Text

PDF

Evaluation of the culture of safety: survey of clinicians and managers in an academic medical center
  1. P J Pronovost1,
  2. B Weast1,
  3. C G Holzmueller1,
  4. B J Rosenstein1,
  5. R P Kidwell1,
  6. K B Haller1,
  7. E R Feroli1,
  8. J B Sexton2,
  9. H R Rubin1
  1. 1The Johns Hopkins Hospital, Baltimore, MD, USA
  2. 2University of Texas, Center of Excellence for Patient Safety Research and Practice, Austin, TX, USA
  1. Correspondence to:
 Dr P J Pronovost
 Department of Anesthesiology and Critical Care Medicine, The Johns Hopkins University, 600 N Wolfe Street, Meyer 295, Baltimore, MD 21287, USA; ppronovojhmi.edu

Abstract

Background: Despite the emphasis on patient safety in health care, few organizations have evaluated the extent to which safety is a strategic priority or their culture supports patient safety. In response to the Institute of Medicine’s report and to an organizational commitment to patient safety, we conducted a systematic assessment of safety at the Johns Hopkins Hospital (JHH) and, from this, developed a strategic plan to improve safety. The specific aims of this study were to evaluate the extent to which the culture supports patient safety at JHH and the extent to which safety is a strategic priority.

Methods: During July and August 2001 we implemented two surveys in disparate populations to assess patient safety. The Safety Climate Scale (SCS) was administered to a sample of physicians, nurses, pharmacists, and other ICU staff. SCS assesses perceptions of a strong and proactive organizational commitment to patient safety. The second survey instrument, called Strategies for Leadership (SLS), evaluated the extent to which safety was a strategic priority for the organization. This survey was administered to clinical and administrative leaders.

Results: We received 395 completed SCS surveys from 82% of the departments and 86% of the nursing units. Staff perceived that supervisors had a greater commitment to safety than senior leaders. Nurses had higher scores than physicians for perceptions of safety. Twenty three completed SLS surveys were received from 77% of the JHH Patient Safety Committee members and 50% of the JHH Management Committee members. Management Committee responses were more positive than Patient Safety Committee, indicating that management perceived safety efforts to be further developed. Strategic planning received the lowest scores from both committees.

Conclusions: We believe this is one of the first large scale efforts to measure institutional culture of safety and then design improvements in health care. The survey results suggest that strategic planning of patient safety needs enhancement. Several efforts to improve our culture of safety were initiated based on these results, which should lead to measurable improvements in patient safety.

  • patient safety
  • culture of safety
  • organizational culture
  • change management
  • leadership

Statistics from Altmetric.com

Despite the awareness that many patients are harmed rather than helped when encountering the healthcare system,1 and that patients receive on average half the treatments they ought to,2 there are few examples of structured efforts to eliminate these ills.3 To improve, organizations must identify broken systems and fix them. Systems include both how we organize work and the culture of organizations.

While most attempts to improve safety in health care are reactive, responding after someone is harmed, efforts to proactively identify and eliminate hazards have the potential to significantly improve safety.4 To accomplish this, patient safety must be viewed as a strategic priority around which the entire efforts of the organization must be focused. The Baldrige framework provides a lens to evaluate the extent to which the efforts of the organization are focused around a strategic priority.5

While the extent to which safety is a strategic priority may reflect attitudes of the leadership toward patient safety, it may not reflect the organization’s culture—the collection of values, beliefs and assumptions that guides members’ behaviors.6–8 Helmreich describes employees in a safe culture as being guided by an organization wide commitment to safety where safety standards are upheld on a personal and team level.8 Characteristics of a strong and proactive safety culture include the commitment of the leadership to discussing and learning from errors, documenting and improving patient safety, encouraging and practising teamwork, spotting potential hazards, using systems for reporting and analyzing adverse events, and celebrating workers as heroes improving safety rather than as villains committing errors.

Recent evidence from aviation supports the association between a culture of safety and reduced pilot error.9 Indeed, much of the improvements in aviation safety are attributed to improved crew coordination, communication, and decision making10,11 The culture in aviation encourages individuals to step forward and share their stories about adverse events. In addition to technical skills, pilots are now hired for their ability to coordinate activities, learn from errors, and recognize that others can contribute to problem solving. Teamwork training is nearly universal in aviation and is formally mandated by the United Nations for all carriers flying international routes.11 Survey results contrasting safety attitudes in aviation and medicine suggest that medicine generally lags behind aviation and needs improvement.11

Despite the emphasis on patient safety in health care, few organizations have evaluated the extent to which safety is a strategic priority or their culture supports patient safety. In response to the Institute of Medicine’s report and to an organizational commitment to patient safety, we conducted a systematic assessment of safety at the Johns Hopkins Hospital (JHH) and, from this, developed a strategic plan to improve safety. The specific aims of this study were to evaluate the extent to which the culture supports patient safety at JHH and the extent to which safety is a strategic priority.

METHODS

The JHH is a 910 bed academic medical center located in an urban setting. During July and August 2001 we implemented two survey instruments in disparate populations to assess patient safety. The Safety Climate Scale (SCS), adapted from the Flight Management Attitudes and Safety Survey,9–12 was used to evaluate safety culture among staff, and the Strategies for Leadership Survey (SLS), developed by the Volunteer Hospital Association Inc (VHA), was used to evaluate the extent to which safety was a strategic priority for hospital leaders.

Safety climate scale (SCS)

The SCS is a 10 item survey which uses a 5-point Likert scale where 1 = disagree, 2 = somewhat disagree, 3 = neutral, 4 = somewhat agree, and 5 = agree. This survey evaluates perceptions of a strong and proactive organizational commitment to patient safety. It taps physician, nurse and senior leadership’s commitment to safety, their knowledge of how to report adverse events, and their understanding of systems as the cause of adverse events.

The SCS was adapted from a widely used survey tool in aviation that queries cockpit management attitudes. The aviation questionnaire was developed to measure attitudes toward stress, status hierarchies, leadership, and interpersonal interaction.13 Scales (as well as individual items) from the health care and aviation surveys have demonstrated good test-retest reliability,14 good internal consistency, and replicable factor structures.12,14 Moreover, they are predictive of pilot performance,9,13 high speed rail incident rates,15 risk adjusted intensive care unit (ICU) patient outcomes,14 and are sensitive to training interventions.16–19

Study sample

The goal of this survey was to evaluate the safety attitudes of staff throughout JHH, thereby providing broad insights into our safety climate. As such, we used convenience sampling to survey a wide range of providers throughout the JHH. Physicians representing each department, nurses representing each clinical nursing unit, pharmacists, respiratory therapists, clinical and pharmacy technicians, nursing aides, clerical staff and others (non-licensed staff and those who did not specify a role) were approached. Our physician sample consisted of 22 physician advisors from each department who sit on the hospital’s Clinical Performance Improvement Committee. Because ICUs are high risk areas20 but do not have physician advisors, we also surveyed ICU physician directors and co-directors from four adult ICUs. The nursing sample consisted of nurse managers from 70 clinical nursing units, as well as other nurses and staff from each unit. The nurse manager randomly selected up to four nurses and two other staff. The pharmacy sample consisted of the assistant director of pharmacy who, in turn, distributed the survey to 96 clinical pharmacists, as well as pharmacy technicians.

Thus, our study sample included clinical leaders (physician advisors, ICU directors, nurse managers and a pharmacy director), clinical staff (nurses on the units, pharmacists, clinical associates, pharmacy technicians), and non-clinical staff (clerical, non-licensed).

Method of data collection

The survey was distributed by email and accompanied by a cover letter with instructions to fax the survey back to the Vice-President for Medical Affairs within 2 weeks. A second mailing was not performed.

Data analysis

The results are presented as the proportion that agreed with each item (survey response 4 or 5). We stratified results of the SCS using the following provider types: physician, nurse, nurse manager, pharmacist, and other. Differences between groups were evaluated using the χ2 test.

Figure 1

Safety climate scale scores at Johns Hopkins Hospital and across 13 airlines. Adapted from Sexton et al12 with permission.

Use of data to inform safety efforts

Scores on individual items of the SCS and variation among types of caregivers would help to focus future patient safety efforts.

Strategies for leadership survey (SLS)

The SLS, developed by VHA Inc and built on the Malcolm Baldrige framework, evaluates the extent to which safety is a strategic priority for an organization.5 This survey was grouped into six core areas: (1) leadership, (2) strategic planning, (3) information and analysis, (4) human resources, (5) process management, and (6) patient and family involvement.

Within each core area, key aspects of safety were evaluated with one or more questions. Responses to each item were rated using a 5-point Likert scale with the following potential responses: 1 = there has been no discussion around this activity; 2 = this activity is under discussion but there is no implementation; 3 = this activity is partially implemented in some or all areas of the organization; 4 = this activity is fully implemented in some areas of the organization; 5 = this activity is fully implemented throughout the organization.

Study sample

The goal of this survey was to evaluate the extent to which patient safety was a strategic priority among senior clinical and administrative leaders. As such, we surveyed members of the Patient Safety Committee to obtain our sample of clinical leaders. This 22 member multidisciplinary committee comprises physician, nursing, administrative, technical, legal, communication, and pharmacy leaders. Some of its members include the Vice President for Medical Affairs, ICU leadership, Coordinator of Nursing Clinical Quality, Deputy Director for Communications and Public Affairs, Director for Medical Staff Administration, Director of Hospital Epidemiology and Infection Control, Assistant Director of Facilities, Assistant Director of Pharmacy, Director of Performance Improvement and Utilization Management, and Chairpersons of the hospital’s ethics committee. Only one person completed both the SCS and SLS.

To obtain our sample of senior administrative leaders, we sent the same SLS to the JHH’s Management Committee. This 12 member multidisciplinary team comprises hospital leaders including the President, Executive Vice President/COO and Vice Presidents.

Method of data collection

The survey was administered by mail with an attached cover letter to the clinical and administrative leaders. In the letter were instructions for completing and returning the survey within 2 weeks to the Vice-President for Medical Affairs. A second mailing was not undertaken.

Data analysis

Results are presented as the proportion that responded 4 or 5 (full or partial implementation) and 1 or 2 (no discussion or implementation). We aggregated the responses to the questions in each core area of the SLS to produce a summary score. Results of the SLS were stratified by membership in the Patient Safety Committee versus Management Committee. Differences between groups were evaluated using a χ2 test.

Use of data to inform safety efforts

The score on the individual domains of the SLS and variation between managers and clinicians would help to inform future patient safety efforts.

RESULTS

Safety climate scale (SCS)

Three hundred and ninety five completed surveys were received from 18 of the 22 (82%) physician advisors and from all eight ICU physicians giving a total of 26 physicians; from 60 of 70 (86%) clinical nursing units giving a total of 198 nurses including 29 managers and 169 staff nurses; from 84 of 96 (88%) clinical pharmacists; and from 87 others that included pharmacy technicians, support associates, unit clerks, non-licensed staff, and role unspecified (table 1).

Table 1

Respondents to Safety Climate Scale (SCS) and Strategies for Leadership Survey (SLS)

The results of the SCS are presented in table 2. The survey results indicated that respondents perceived a stronger commitment to safety from their direct supervisors than from senior leaders in the organization. In addition, physicians gave lower scores than nurses for most questions. In particular, fewer physicians (54%) than nurses (84%) perceived encouragement from their supervisors to report safety concerns (question 5). Fewer physicians (46%) than nurses (86%) indicated they were aware of the proper channels to report adverse events (question 6). In addition, fewer physicians (54%) than nurses (63%) were aware that patient safety was a major initiative for the hospital (question 9).

Table 2

Percentage of respondents who agreed with questions in the Safety Climate Scale in aviation and in our survey stratified by provider type

Strategies for leadership survey (SLS)

The characteristics of respondents to the SLS are shown in table 1. We received surveys from 17 of 22 (77%) members of the Patient Safety Committee and from six of 12 (50%) members of the Management Committee. The overall mean score from both committees was 162 out of a possible 270 with the Management Committee scoring 186 and the Patient Safety Committee scoring 153.

There were statistically significant differences between members of the Management and Patient Safety Committees for eight of the nine questions (89%). The proportion of respondents to the SLS who perceived safety areas as either fully or partially implemented is presented in table 3. Members of the Management Committee provided higher scores than members of the Patient Safety Committee in each of the core areas, indicating their perception that safety in that core area is further developed. Few respondents perceived that strategic planning was fully or partially implemented. In addition, the relatively low scoring for leadership by the Patient Safety Committee is consistent with the staff’s perceptions of senior leaders in the SCS.

Table 3

Percentage of respondents to the Strategies for Leadership Survey (SLS) who perceived full or partial implementation of the safety item: stratified by Patient Safety Committee and Management Committee*

Table 4 shows the proportion of respondents who perceived that safety is not discussed or implemented (responded 1 or 2). Most of these respondents felt that strategic planning for safety was not discussed or implemented.

Table 4

Percentage of respondents to the Strategies for Leadership Survey (SLS) who perceived no discussion or implementation plan for each item: stratified by Patient Safety Committee and Management Committee*

DISCUSSION

We conducted an assessment of the culture of safety and the extent to which safety is a strategic priority throughout the JHH by administering two survey instruments to a diverse group of professionals. The main finding of the SCS, which evaluated culture of safety, was that staff generally believed their direct supervisors were more committed to safety than senior hospital leaders. This finding was especially true for nurses. In addition, we found that physicians were less aware of patient safety initiatives than nurses. Nurses rated the culture of safety higher than physicians for most questions on the staff survey.

The major findings of the SLS, which evaluated the extent to which safety is a strategic priority, were a lack of strategic planning for patient safety and differences in perceptions of safety between members of the Patient Safety and Management Committees. Members of the Management Committee felt they were doing a better job on safety than did members of the Patient Safety Committee. Furthermore, members of both the Patient Safety and Management Committees felt that systems to allow collaboration and sharing of best practices were poorly developed and that greater efforts were needed in human resources to create reward and recognition systems for patient safety.

Taken together, these surveys highlight the following specific opportunities for improving patient safety:

  • Senior leaders need to become more visible to front line staff in their efforts to improve patient safety.

  • There should be a proactive strategic planning process for patient safety.

  • Greater efforts are needed to involve and educate physicians about patient safety.

We believe the perceived lack of strategic planning for safety reflects the activity of the leadership of the institution in safety. To date, most safety efforts have focused on reacting to crises rather than proactively identifying hazards and improving systems. It is believed that the physicians’ lack of knowledge for reporting adverse events is also a reflection of institutional efforts, as most of the efforts to enhance reporting of medication errors have been led by nurses and pharmacists.

Results from the SLS have strengthened efforts to improve safety. These results were presented to the Performance Improvement Council, the Management Committee, the Clinical Performance Improvement Committee (comprised of physician advisors), nursing leaders, and performance improvement/utilization management meetings.

To a large extent, the results from both surveys have prompted the Patient Safety Committee to develop a strategic plan to improve patient safety. The cornerstone of this plan was to develop and implement a comprehensive unit based patient safety program (CUSP) that could be sequentially implemented in the work units (for example, clinical nursing unit) with front line staff involvement. This program attempts to improve the culture of safety in the units and in the institution and to lead to measurable improvements in safety. The infrastructure of CUSP affords clinical staff the opportunity to assess their particular and often unique environment and to identify opportunities to reduce system errors and to engage senior executive staff in obtaining resources for system improvements. This program, meant to be implemented at the nursing unit level, contrasts with the centralized approach used to improve safety at the Veterans Affairs (VA) Health System.21

The Patient Safety Committee has also developed several other initiatives to improve safety. This committee has created a safety mission statement, developed a non-punitive error reporting policy, created an information sheet of safety tips for patients and families to follow to help ensure their safety while in the hospital, educated staff on the science of safety and how to disclose errors, developed a patient safety intranet site on which staff can share stories regarding patient safety, and implemented senior executive walk rounds where senior leaders “adopt” a functional unit. We are currently pilot testing the senior leader unit adoption program in ICUs and operating rooms at the JHH. In addition, we participated in the Institute for Healthcare Improvement’s “Quantum Leaps in Patient Safety” effort. Table 5 lists the initiatives that have resulted from our cultural assessment. To evaluate the impact of these improvement efforts, we plan to repeat the cultural assessment in one year.

Table 5

Patient safety initiatives at the Johns Hopkins Hospital (JHH) as a result of the cultural and leadership assessments

A major initiative was to educate staff on the science related to safety. This initiative has resulted in safety talks at the Dean’s retreat comprising department chairs and other senior executives; the Johns Hopkins Medicine Strategic Planning retreat of leaders throughout Johns Hopkins Medicine; grand rounds in 10 academic departments; the Nursing Leadership Forum which comprised nursing unit managers; and in-services to nurses on approximately 15 nursing units. Education on the science of safety has enhanced the capacity of our staff to focus on system factors related to safety.

There are, however, several limitations to our study. Firstly, we did not formally evaluate the performance properties of the survey instruments. Nonetheless, both surveys have been widely used, pilot tested, and refined. The second limitation is the potential for bias in the sampling frame. We sought input from a diverse group of professionals at the JHH (across departments and clinical nursing units) in the hope of obtaining an accurate view of our organizational culture. Nonetheless, because we did not randomly select staff, we may have introduced selection bias. As a result, we were unable to investigate rigorously the extent of within-group variability in perceptions of safety climate within a provider type or work unit (otherwise known as “climate strength”22). In addition, because the safety culture survey was administered by the committee seeking the information, the responses may be more positive than if a third party had implemented this tool. Nonetheless, the pre- and post-intervention surveys were administered the same way, with post-intervention scores being significantly higher than pre-intervention scores. We do not have data available to compare survey respondents to all physicians and staff at Johns Hopkins. The sample also did not include house staff physicians who are primarily responsible for many aspects of patient care. Moreover, comparative data from additional healthcare settings (academic and otherwise) would have provided a more direct context to our findings.

By evaluating our safety climate relative to commercial aviation, another high reliability industry, we have demonstrated the potential of healthcare settings to fulfil the high expectations of safety set by our patients. The positive results achieved could be attributable to a number of factors. Firstly, the fiercely academic and competitive culture at the JHH could be translated into attempts to be competitively safe (informal feedback from providers supports this notion). Also, internal initiatives could be responsible for the high safety climate scores as the JHH has made substantial efforts to improve patient safety over the last 5 years, with a particular focus on improving medication safety and reporting medication errors. In addition, some of our strong safety climate scores could be attributed to the aforementioned selection bias and/or sampling bias (as response rates go up, responses generally become more negative because negative respondents typically avoid surveys). With respect to these biases, higher response rates from current and additional provider types may have produced less positive safety climate results. However, the very positive safety climate at the JHH is encouraging, and we hope to build on this enthusiasm and trust with additional efforts towards improving patient safety.

In summary, we believe this is one of the first large scale efforts within one healthcare institution to measure the culture of safety and the extent to which safety is a strategic priority and, based on these findings, to design improvements informed by input from the front line. The results suggest that enhanced efforts are needed to improve our culture of safety and our focus on strategic planning for patient safety. As a result, we have initiated several efforts to improve our culture of safety and to enhance safety as a strategic priority. We look forward to collaboration and comments as we work to transform our efforts to improve patient safety from rhetoric to reality.

Key messages

  • Senior leaders need to become more visible to front line staff in their efforts to improve patient safety.

  • There should be a proactive strategic planning process for patient safety.

  • Physicians were less aware of patient safety efforts than nurses.

  • More work is needed to involve and educate physicians about patient safety efforts.

REFERENCES

View Abstract

Footnotes

  • Funded in part by the Agency for Healthcare Research and Quality, grant number U18HS11902-01.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles

  • Action points
    Tim Albert