METHODS

The JHH is a 910 bed academic medical center located in an urban setting. During July and August 2001 we implemented two survey instruments in disparate populations to assess patient safety. The Safety Climate Scale (SCS), adapted from the Flight Management Attitudes and Safety Survey,^9–12 was used to evaluate safety culture among staff, and the Strategies for Leadership Survey (SLS), developed by the Volunteer Hospital Association Inc (VHA), was used to evaluate the extent to which safety was a strategic priority for hospital leaders.

Safety climate scale (SCS)

The SCS is a 10 item survey which uses a 5-point Likert scale where 1 = disagree, 2 = somewhat disagree, 3 = neutral, 4 = somewhat agree, and 5 = agree. This survey evaluates perceptions of a strong and proactive organizational commitment to patient safety. It taps physician, nurse and senior leadership’s commitment to safety, their knowledge of how to report adverse events, and their understanding of systems as the cause of adverse events.

The SCS was adapted from a widely used survey tool in aviation that queries cockpit management attitudes. The aviation questionnaire was developed to measure attitudes toward stress, status hierarchies, leadership, and interpersonal interaction.¹³ Scales (as well as individual items) from the health care and aviation surveys have demonstrated good test-retest reliability,¹⁴ good internal consistency, and replicable factor structures.^12,14 Moreover, they are predictive of pilot performance,^9,13 high speed rail incident rates,¹⁵ risk adjusted intensive care unit (ICU) patient outcomes,¹⁴ and are sensitive to training interventions.^16–19

Study sample

The goal of this survey was to evaluate the safety attitudes of staff throughout JHH, thereby providing broad insights into our safety climate. As such, we used convenience sampling to survey a wide range of providers throughout the JHH. Physicians representing each department, nurses representing each clinical nursing unit, pharmacists, respiratory therapists, clinical and pharmacy technicians, nursing aides, clerical staff and others (non-licensed staff and those who did not specify a role) were approached. Our physician sample consisted of 22 physician advisors from each department who sit on the hospital’s Clinical Performance Improvement Committee. Because ICUs are high risk areas²⁰ but do not have physician advisors, we also surveyed ICU physician directors and co-directors from four adult ICUs. The nursing sample consisted of nurse managers from 70 clinical nursing units, as well as other nurses and staff from each unit. The nurse manager randomly selected up to four nurses and two other staff. The pharmacy sample consisted of the assistant director of pharmacy who, in turn, distributed the survey to 96 clinical pharmacists, as well as pharmacy technicians.

Thus, our study sample included clinical leaders (physician advisors, ICU directors, nurse managers and a pharmacy director), clinical staff (nurses on the units, pharmacists, clinical associates, pharmacy technicians), and non-clinical staff (clerical, non-licensed).

Method of data collection

The survey was distributed by email and accompanied by a cover letter with instructions to fax the survey back to the Vice-President for Medical Affairs within 2 weeks. A second mailing was not performed.

Data analysis

The results are presented as the proportion that agreed with each item (survey response 4 or 5). We stratified results of the SCS using the following provider types: physician, nurse, nurse manager, pharmacist, and other. Differences between groups were evaluated using the χ² test.

Use of data to inform safety efforts

Scores on individual items of the SCS and variation among types of caregivers would help to focus future patient safety efforts.

Strategies for leadership survey (SLS)

The SLS, developed by VHA Inc and built on the Malcolm Baldrige framework, evaluates the extent to which safety is a strategic priority for an organization.⁵ This survey was grouped into six core areas: (1) leadership, (2) strategic planning, (3) information and analysis, (4) human resources, (5) process management, and (6) patient and family involvement.

Within each core area, key aspects of safety were evaluated with one or more questions. Responses to each item were rated using a 5-point Likert scale with the following potential responses: 1 = there has been no discussion around this activity; 2 = this activity is under discussion but there is no implementation; 3 = this activity is partially implemented in some or all areas of the organization; 4 = this activity is fully implemented in some areas of the organization; 5 = this activity is fully implemented throughout the organization.

Study sample

The goal of this survey was to evaluate the extent to which patient safety was a strategic priority among senior clinical and administrative leaders. As such, we surveyed members of the Patient Safety Committee to obtain our sample of clinical leaders. This 22 member multidisciplinary committee comprises physician, nursing, administrative, technical, legal, communication, and pharmacy leaders. Some of its members include the Vice President for Medical Affairs, ICU leadership, Coordinator of Nursing Clinical Quality, Deputy Director for Communications and Public Affairs, Director for Medical Staff Administration, Director of Hospital Epidemiology and Infection Control, Assistant Director of Facilities, Assistant Director of Pharmacy, Director of Performance Improvement and Utilization Management, and Chairpersons of the hospital’s ethics committee. Only one person completed both the SCS and SLS.

To obtain our sample of senior administrative leaders, we sent the same SLS to the JHH’s Management Committee. This 12 member multidisciplinary team comprises hospital leaders including the President, Executive Vice President/COO and Vice Presidents.

Method of data collection

The survey was administered by mail with an attached cover letter to the clinical and administrative leaders. In the letter were instructions for completing and returning the survey within 2 weeks to the Vice-President for Medical Affairs. A second mailing was not undertaken.

Data analysis

Results are presented as the proportion that responded 4 or 5 (full or partial implementation) and 1 or 2 (no discussion or implementation). We aggregated the responses to the questions in each core area of the SLS to produce a summary score. Results of the SLS were stratified by membership in the Patient Safety Committee versus Management Committee. Differences between groups were evaluated using a χ² test.

Use of data to inform safety efforts

The score on the individual domains of the SLS and variation between managers and clinicians would help to inform future patient safety efforts.

RESULTS

Safety climate scale (SCS)

Three hundred and ninety five completed surveys were received from 18 of the 22 (82%) physician advisors and from all eight ICU physicians giving a total of 26 physicians; from 60 of 70 (86%) clinical nursing units giving a total of 198 nurses including 29 managers and 169 staff nurses; from 84 of 96 (88%) clinical pharmacists; and from 87 others that included pharmacy technicians, support associates, unit clerks, non-licensed staff, and role unspecified (table 1).

The results of the SCS are presented in table 2. The survey results indicated that respondents perceived a stronger commitment to safety from their direct supervisors than from senior leaders in the organization. In addition, physicians gave lower scores than nurses for most questions. In particular, fewer physicians (54%) than nurses (84%) perceived encouragement from their supervisors to report safety concerns (question 5). Fewer physicians (46%) than nurses (86%) indicated they were aware of the proper channels to report adverse events (question 6). In addition, fewer physicians (54%) than nurses (63%) were aware that patient safety was a major initiative for the hospital (question 9).

Strategies for leadership survey (SLS)

The characteristics of respondents to the SLS are shown in table 1. We received surveys from 17 of 22 (77%) members of the Patient Safety Committee and from six of 12 (50%) members of the Management Committee. The overall mean score from both committees was 162 out of a possible 270 with the Management Committee scoring 186 and the Patient Safety Committee scoring 153.

There were statistically significant differences between members of the Management and Patient Safety Committees for eight of the nine questions (89%). The proportion of respondents to the SLS who perceived safety areas as either fully or partially implemented is presented in table 3. Members of the Management Committee provided higher scores than members of the Patient Safety Committee in each of the core areas, indicating their perception that safety in that core area is further developed. Few respondents perceived that strategic planning was fully or partially implemented. In addition, the relatively low scoring for leadership by the Patient Safety Committee is consistent with the staff’s perceptions of senior leaders in the SCS.

Table 4 shows the proportion of respondents who perceived that safety is not discussed or implemented (responded 1 or 2). Most of these respondents felt that strategic planning for safety was not discussed or implemented.

DISCUSSION

We conducted an assessment of the culture of safety and the extent to which safety is a strategic priority throughout the JHH by administering two survey instruments to a diverse group of professionals. The main finding of the SCS, which evaluated culture of safety, was that staff generally believed their direct supervisors were more committed to safety than senior hospital leaders. This finding was especially true for nurses. In addition, we found that physicians were less aware of patient safety initiatives than nurses. Nurses rated the culture of safety higher than physicians for most questions on the staff survey.

The major findings of the SLS, which evaluated the extent to which safety is a strategic priority, were a lack of strategic planning for patient safety and differences in perceptions of safety between members of the Patient Safety and Management Committees. Members of the Management Committee felt they were doing a better job on safety than did members of the Patient Safety Committee. Furthermore, members of both the Patient Safety and Management Committees felt that systems to allow collaboration and sharing of best practices were poorly developed and that greater efforts were needed in human resources to create reward and recognition systems for patient safety.

Taken together, these surveys highlight the following specific opportunities for improving patient safety:

• Senior leaders need to become more visible to front line staff in their efforts to improve patient safety.

• There should be a proactive strategic planning process for patient safety.

• Greater efforts are needed to involve and educate physicians about patient safety.

We believe the perceived lack of strategic planning for safety reflects the activity of the leadership of the institution in safety. To date, most safety efforts have focused on reacting to crises rather than proactively identifying hazards and improving systems. It is believed that the physicians’ lack of knowledge for reporting adverse events is also a reflection of institutional efforts, as most of the efforts to enhance reporting of medication errors have been led by nurses and pharmacists.

Results from the SLS have strengthened efforts to improve safety. These results were presented to the Performance Improvement Council, the Management Committee, the Clinical Performance Improvement Committee (comprised of physician advisors), nursing leaders, and performance improvement/utilization management meetings.

To a large extent, the results from both surveys have prompted the Patient Safety Committee to develop a strategic plan to improve patient safety. The cornerstone of this plan was to develop and implement a comprehensive unit based patient safety program (CUSP) that could be sequentially implemented in the work units (for example, clinical nursing unit) with front line staff involvement. This program attempts to improve the culture of safety in the units and in the institution and to lead to measurable improvements in safety. The infrastructure of CUSP affords clinical staff the opportunity to assess their particular and often unique environment and to identify opportunities to reduce system errors and to engage senior executive staff in obtaining resources for system improvements. This program, meant to be implemented at the nursing unit level, contrasts with the centralized approach used to improve safety at the Veterans Affairs (VA) Health System.²¹

The Patient Safety Committee has also developed several other initiatives to improve safety. This committee has created a safety mission statement, developed a non-punitive error reporting policy, created an information sheet of safety tips for patients and families to follow to help ensure their safety while in the hospital, educated staff on the science of safety and how to disclose errors, developed a patient safety intranet site on which staff can share stories regarding patient safety, and implemented senior executive walk rounds where senior leaders “adopt” a functional unit. We are currently pilot testing the senior leader unit adoption program in ICUs and operating rooms at the JHH. In addition, we participated in the Institute for Healthcare Improvement’s “Quantum Leaps in Patient Safety” effort. Table 5 lists the initiatives that have resulted from our cultural assessment. To evaluate the impact of these improvement efforts, we plan to repeat the cultural assessment in one year.

A major initiative was to educate staff on the science related to safety. This initiative has resulted in safety talks at the Dean’s retreat comprising department chairs and other senior executives; the Johns Hopkins Medicine Strategic Planning retreat of leaders throughout Johns Hopkins Medicine; grand rounds in 10 academic departments; the Nursing Leadership Forum which comprised nursing unit managers; and in-services to nurses on approximately 15 nursing units. Education on the science of safety has enhanced the capacity of our staff to focus on system factors related to safety.

There are, however, several limitations to our study. Firstly, we did not formally evaluate the performance properties of the survey instruments. Nonetheless, both surveys have been widely used, pilot tested, and refined. The second limitation is the potential for bias in the sampling frame. We sought input from a diverse group of professionals at the JHH (across departments and clinical nursing units) in the hope of obtaining an accurate view of our organizational culture. Nonetheless, because we did not randomly select staff, we may have introduced selection bias. As a result, we were unable to investigate rigorously the extent of within-group variability in perceptions of safety climate within a provider type or work unit (otherwise known as “climate strength”²²). In addition, because the safety culture survey was administered by the committee seeking the information, the responses may be more positive than if a third party had implemented this tool. Nonetheless, the pre- and post-intervention surveys were administered the same way, with post-intervention scores being significantly higher than pre-intervention scores. We do not have data available to compare survey respondents to all physicians and staff at Johns Hopkins. The sample also did not include house staff physicians who are primarily responsible for many aspects of patient care. Moreover, comparative data from additional healthcare settings (academic and otherwise) would have provided a more direct context to our findings.

By evaluating our safety climate relative to commercial aviation, another high reliability industry, we have demonstrated the potential of healthcare settings to fulfil the high expectations of safety set by our patients. The positive results achieved could be attributable to a number of factors. Firstly, the fiercely academic and competitive culture at the JHH could be translated into attempts to be competitively safe (informal feedback from providers supports this notion). Also, internal initiatives could be responsible for the high safety climate scores as the JHH has made substantial efforts to improve patient safety over the last 5 years, with a particular focus on improving medication safety and reporting medication errors. In addition, some of our strong safety climate scores could be attributed to the aforementioned selection bias and/or sampling bias (as response rates go up, responses generally become more negative because negative respondents typically avoid surveys). With respect to these biases, higher response rates from current and additional provider types may have produced less positive safety climate results. However, the very positive safety climate at the JHH is encouraging, and we hope to build on this enthusiasm and trust with additional efforts towards improving patient safety.

In summary, we believe this is one of the first large scale efforts within one healthcare institution to measure the culture of safety and the extent to which safety is a strategic priority and, based on these findings, to design improvements informed by input from the front line. The results suggest that enhanced efforts are needed to improve our culture of safety and our focus on strategic planning for patient safety. As a result, we have initiated several efforts to improve our culture of safety and to enhance safety as a strategic priority. We look forward to collaboration and comments as we work to transform our efforts to improve patient safety from rhetoric to reality.