Article Text
PDF
Abstract
Objectives To assess the relationship between changes in clinician attitude and changes in postoperative outcomes following a checklist-based surgical safety intervention.
Design Pre- and post intervention survey.
Setting Eight hospitals participating in a trial of a WHO surgical safety checklist.
Participants Clinicians actively working in the designated study operating rooms at the eight hospitals.
Survey instrument Modified operating-room version Safety Attitudes Questionnaire (SAQ).
Main outcome measures Change in mean safety attitude score and correlation between change in safety attitude score and change in postoperative outcomes, plus clinician opinion of checklist efficacy and usability.
Results Clinicians in the preintervention phase (n=281) had a mean SAQ score of 3.91 (on a scale of 1 to 5, with 5 representing better safety attitude), while the postintervention group (n=257) had a mean of 4.01 (p=0.0127). The degree of improvement in mean SAQ score at each site correlated with a reduction in postoperative complication rate (r=0.7143, p=0.0381). The checklist was considered easy to use by 80.2% of respondents, while 19.8% felt that it took a long time to complete, and 78.6% felt that the programme prevented errors. Overall, 93.4% would want the checklist used if they were undergoing operation.
Conclusions Improvements in postoperative outcomes were associated with improved perception of teamwork and safety climate among respondents, suggesting that changes in these may be partially responsible for the effect of the checklist. Clinicians held the checklist in high regard and the overwhelming majority would want it used if they were undergoing surgery themselves.
- Communication
- healthcare quality improvement
- patient safety
- safety culture
- surgery
Statistics from Altmetric.com
Delivery of operative care requires the coordination of skilled clinicians from different disciplines working together in a time-sensitive, complex environment. As surgery plays an increasingly prominent role in healthcare around the world, growing attention is being focused on the safety and quality of such care. A growing body of research has linked teamwork, safety culture and climate, and patient outcomes following surgery.1–5 Using lessons learnt from other high-reliability industries such as aviation, interventions have been designed to address gaps in patient safety.6 7 Many such interventions focus on teamwork and safety climate, yet the relationship between safety climate and patient outcomes is not well understood.8
We recently reported the results of a checklist-based intervention designed by the Safe Surgery Saves Lives initiative of the WHO to reduce surgical complications globally.7 This intervention was associated with a greater than one-third reduction in postoperative complications and deaths in hospitals located in diverse geographic and economic environments (Canada, India, Jordan, New Zealand, Philippines, Tanzania, UK and USA). However, unlike many interventions involving drugs or medical devices, application of a team-based intervention necessitates a change in clinical behaviour in order to be successful. Study of these changes requires qualitative and quantitative methods to understand how the different factors involved in patient care relate to improved outcomes. Brown et al suggest that a thorough evaluation of quality improvement interventions requires the use of ‘mixed methods,’ and the study of multiple endpoints including outcomes, fidelity to the intervention and intervening variables, such as changes in the processes of care and team function.9 10 In this study, we sought to understand some of the attitudes and perceptions of the clinical teams involved in implementation of the WHO Surgical Safety Checklist. We hypothesise that there is a relationship between changes in teamwork and safety climate and the magnitude of the checklist’s effect on postoperative outcomes.
Methods
Survey instrument
The Safety Attitudes Questionnaire (SAQ) is a validated instrument used to measure attitudes and perceptions in various safety-related domains in healthcare.11–13 A modification has been developed for use in the operating rooms (OR), from which we selected six items related to the teamwork climate and safety climate domains, areas that we felt most likely to be relevant to the checklist intervention. These items were ‘I would feel safe being treated here as a patient,’ ‘Briefing OR personnel before a surgical procedure is important for patient safety,’ ‘I am encouraged by my colleagues to report any safety concerns I may have,’ ‘In the ORs here, it is difficult to speak up if I perceive a problem with patient care,’ ‘The physicians and nurses here work together as a well-coordinated team,’ and ‘Personnel frequently disregard rules or guidelines that are established for the OR.’ This survey instrument was chosen because of its well-established validity and previous use in research to better understand issues of safety and teamwork in the surgical environment. Additionally, as a surgery-specific modification had previously been developed, we were able to select from these items for our study. A panel of experts in surgical patient safety developed six additional items specifically related to the checklist intervention (‘The checklist was easy to use,’ ‘The checklist improved OR safety,’ ‘The checklist took a long time to complete,’ ‘If I were having an operation, I would want the checklist to be used,’ ‘Communication was improved through use of the checklist,’ and ‘The checklist helped prevent errors in the OR’) which were included only in the postintervention version of the survey. Responses to these items were not included in the calculation of the mean SAQ score. All responses were recorded on a five-point Likert scale (1=disagree strongly, 2=disagree slightly, 3=neutral, 4=agree slightly, 5=agree strongly). In those instances where no answer was provided for a specific item, the item was scored as neutral, although non-response rates were measured. Three items (‘In the ORs here…,’ ‘Personnel frequently disregard…’ and ‘The checklist took a long time…’) were reverse-scored to calculate summary statistics.
Data collection
The preintervention survey was distributed during 2 weeks of the baseline data-collection period of the pilot study of the WHO Surgical Safety Checklist described elsewhere.7 The postintervention survey was distributed in a similar fashion over a 2-week period at the end of the intervention phase prior to reporting of the results of the study. The questionnaire was distributed in the study ORs over a 2-week period to staff working in those ORs, targeting those clinicians with experience using the checklist. The survey was not limited to any particular discipline within the OR and was specifically targeted to surgeons, anaesthesia professionals, nurses, surgical technologists and trainees in the above fields. Under the terms of our human subjects’ committee approval, surveys were collected in envelopes placed in the room to preserve anonymity, although this inhibited calculation of the response rate. A specific item on the data form ascertained whether the respondent had previously completed the survey, and the instructions on the form asked that each participant only fill out the form once per phase. The local data coordinator gathered the returned surveys and submitted the results in an electronic form to the study team.
Postsurgical inpatient morbidity and mortality data were collected prospectively through regular chart monitoring and communication with clinical teams. We utilised definitions of complications from the American College of Surgeons National Surgical Quality Improvement Project: acute renal failure, bleeding requiring ≥4 units of red cell transfusion within 72 h after surgery, cardiac arrest requiring cardiopulmonary resuscitation, coma for ≥24 h, deep venous thrombosis, myocardial infarction, unplanned intubation, ventilator use for ≥48 h, pneumonia, pulmonary embolism, stroke, major wound disruption, surgical site infection, sepsis, septic shock, systemic inflammatory response syndrome, unplanned return to the OR, vascular graft failure and death.14 Data collection continued until patients were discharged from the hospital or 30 days had elapsed, whichever occurred first. Details of the rates of death and complications following operation during the study have been reported elsewhere, as have the specifics of the checklist-based implementation programme.7
The study was approved by the Human Subjects Committee of the Harvard School of Public Health, as well as by the equivalent body at each of the study sites. No incentives were provided for return of the survey, and all demographic questions were explicitly optional to assist in protecting anonymity.
Statistical analysis
Respondent characteristics were summarised using proportions for discrete variables, average scores for ordered categorical variables and means with standard deviations and ranges for continuous variables. The Wilcoxon rank-sum test, adjusted for site, was used for comparing continuous or ordered categorical variables between the pre- and postintervention periods.15 The Cronbach α was calculated for the six items taken from the SAQ to assess for internal consistency. Mantel–Haenszel tests, stratified by site, were used to compare discrete variables between the pre- and postintervention periods. Associations between pairs of site level variables (eg, change in mean safety attitude score versus relative reduction in complications at the site level) were estimated using the Spearman correlation coefficients. Alpha was set at 0.05, and all p values were two-sided. SAS version 9.1 was used for all analyses.
Results
Clinicians from seven of the eight sites completed the SAQ during the two phases of the study, with 281 respondents prior to checklist implementation and 257 after. One site failed to distribute the questionnaires. The mean age of respondents was 38.0 years in the preintervention group and 37.5 after. The median time in current job was 2–5 years in both groups, and the median experience in healthcare was 2–5 years among baseline respondents and 5–10 years after the intervention. Demographics of the respondents are displayed in table 1. The two groups differed significantly only in the distribution of respondents among the sites (p=0.0054), with other differences in demographics being non-significant (p>0.05). The missing response rate for the SAQ was 0.3%, with nine of 1686 items unanswered in the preintervention cohort and one of 1542 following checklist implementation. No forms were returned with an indication that the respondent had previously completed the survey.
Demographics of respondents
As shown in table 2, the aggregate mean safety attitude score rose from 3.91 to 4.01 following implementation of the checklist (p=0.0127), with higher response rates suggesting improved teamwork and safety climates. At an individual item level, two of the statements—‘Briefing OR personnel before a surgical procedure is important for patient safety’ and ‘I am encouraged by my colleagues to report any safety concerns I may have’—showed a significantly stronger agreement postintervention (p=0.0058 and p=0.0225 respectively), while changes in the other four items did not reach statistical significance. The Cronbach α for the six items taken from the SAQ is 0.60. Figure 1 shows the relationship between the change in mean safety attitude score at individual sites and the relative reduction in complications seen at that site (r=0.7143, p=0.0381).
Pre- and postintervention safety attitudes
Changes in Outcome and Changes in Safety Attitude. Each point represents one of the seven sites with complete data on safety attitudes. The x-axis represents the change in mean Safety Attitudes Questionnaire (SAQ) score following the intervention, in points on the Likert scale. The y-axis represents the percentage reduction in complications observed following intervention. The Spearman coefficient is 0.7143 with p=0.0381.
Of the 257 clinicians who completed the postintervention survey, 80.2% thought the checklist was easy to use, while only 19.8% felt it took a long time to complete (table 3). A majority agreed that the checklist improved OR safety and communication (80.2% and 84.8% respectively), and 78.6% thought that the checklist helped prevent errors in the OR. A large majority (93.4%) would want the checklist used if they were having an operation; only four respondents (1.6%) disagreed with this statement.
Clinician opinion of the checklist (N=257)
Discussion
Implementation of the WHO Safe Surgery Saves Lives checklist-based quality improvement project was associated with a small but significant increase in mean teamwork and safety climate score among operating personnel. Positive changes in perception of teamwork and safety climate by these clinicians correlated with the degree of improvement in postoperative morbidity and mortality. Furthermore, nearly all respondents expressed that they would want the checklist used if they were undergoing an operation, suggesting that the checklist had face validity to frontline clinicians. The improvements in outcomes observed in the study of the WHO Surgical Safety Checklist were striking, with rates of complication decreasing from 11% during the baseline period to 7% following implementation. Improvements in adherence to selected safety indicators were also significant. However, the improvements in outcomes did not correlate consistently with improvements in specific processes of care, suggesting that an intervening mechanism was at work.7 In addition to the components of the checklist that function to trap omission of specific safety steps, many items were designed to enhance team function and communication in the OR.16 Variation in improvement of team function and safety climate are possible explanations for some of the differences in improvement observed among the sites, and this study suggests a possible explanation for this effect.
Data from previous studies have linked introductions, briefings and improved teamwork in the OR with improved safety climate.12 13 17 Observational studies have shown an association between good teamwork, as marked by frequent briefing, information sharing, enquiry, vigilance and awareness, and decreased risk of postoperative complications.4 Dixon-Woods, in a recent review of ethnographc study of OR processes concluded that major barriers to patient safety were present on both a structural and cultural level.18 Institutions whose frontline workers and managers score higher on safety climate surveys have been found to have lower rates of adverse patient safety indicators, as defined by the Agency for Healthcare Research and Quality.5 Our findings support this previous work, with the additional demonstration that a checklist-based quality-improvement programme can improve both outcomes and safety climate in tandem.
Clinicians who used the checklist in the OR had a generally positive view of the tool. They did not find it difficult to use and perceived its value in ensuring patient safety. Perhaps most tellingly, nearly all responded that they would want the checklist used in their own care, including some who explicitly doubted the effect of the intervention. Even when clinicians express some scepticism of this quality-improvement tool, this fundamental perception of its value in providing safe care suggests that a well-designed implementation programme can be successful in achieving clinician acceptance and use of the checklist.
There are several limitations to this study. We were unable to track the survey response rate due to the restrictions of our survey methods under human-subjects committee oversight. Without this information, we cannot know whether the respondents are representative of the site as a whole. However, the demographics of the respondents in the preintervention and postintervention surveys suggest that the two groups are very similar. We also ensured that there were no multiple responses to the survey. Additionally, the pilot sites, while geographically and economically diverse, were not chosen to be representative of hospitals providing surgical care. All volunteered to participate in this project and had leadership committed to improving patient safety. Therefore, the findings may not be generalisable to all institutions. Additionally, the survey was administered in the context of a larger quality improvement and research project. Clinicians in the study rooms, as well as the hospitals as a whole, were aware that this initiative was taking place and that the checklist was being studied as it was implemented. There may have been a desire to provide responses that demonstrated success of the programme. However, the variation between the sites, as well as the correlation with efficacy of the checklist in preventing postoperative adverse events, suggests that there was a real relationship between teamwork and safety climate and patient outcomes. A further study of safety attitude in settings where the checklist is being implemented as a quality-improvement project outside a research protocol would be useful to understand this relationship more clearly.
We cannot determine if the association between improvements in safety attitude score and reduction in postoperative complications is a causal relationship, or what the direction of causality might be. The postintervention survey was administered prior to the final analysis and release of the outcome data. However, frontline clinical workers may have independently recognised the changes in outcome that followed implementation of the checklist, and this could have conceivably influenced their perception of the safety climate at their institution. Conversely, the checklist and associated programme may have led to increased communication and thus improved perception of teamwork and safety climate; some other, unidentified factors may have led to the improvements in both attitude and outcome. Additionally, while the change in safety attitude score was statistically significant, the magnitude was small, and further confirmatory research is necessary to explore this relationship in greater depth.
While the SAQ is a well-validated psychometric instrument, we administered only 10% of the 58 questions on the OR-based version, as the longer version was impractical in this large, international quality improvement and research programme, particularly in poorer settings with limited research funds. The six items were selected from the two domains, safety climate and teamwork climate, felt to reflect the safety changes associated with the checklist intervention. While we observed significant changes in the six items administered in these two domains, it is unknown whether the observed effect and association with changes in outcome would remain had the entire instrument been administered. There was acceptable internal consistency among our questions measuring safety climate and teamwork climate.
In this study, we sought to understand the clinician perception of safety climate among OR teams along with attitude towards a specific intervention, the WHO Surgical Safety Checklist. The results of this survey suggest that perception of team function and safety climate increases following introduction of the WHO checklist-based intervention and that this change is correlated with improvements in outcomes. It is possible that some portion of the effect of the intervention is mediated by team attitudes towards safety. Further research will help to understand the strength and mechanism of these relationships.
Acknowledgments
The authors would like to thank S Singer for her advice and guidance.
References
Footnotes
Safe Surgery Saves Lives Study Group: Amman: AS Breizat, AF Awamleh, OG Sadieh; Auckland: AF Merry, SJ Mitchell, V Cochrane, A-M Wilkinson, J Windsor, N Robertson, N Smith, W Guthrie, V Beavis; Ifakara: P Kibatala, B Jullu, R Mayoka, M Kasuga, W Sawaki, N Pak; London: A Darzi, K Moorthy, A Vats, R Davies, K Nagpal, M Sacks; Manila: G Herbosa, T Herbosa, MCM Lapitan, C Meghrajani; New Delhi: S Joseph, A Kumar, H Singh Chauhan; Seattle: EP Dellinger, K Gerber; Toronto RK Reznick, B Taylor, A Slater; Boston: WR Berry, AA Gawande, AB Haynes, SR Lipsitz, TG Weiser; Geneva: L Donaldson, G Dziekan, P Philip; Baltimore: M Makary; Ankara: I Sayek, Sydney B Barraclough.
Funding This study was funded by the WHO. The WHO had no role in study design, analysis or interpretation of data, or on the decision to publish.
Competing interests None.
Ethics approval The study was approved by the Human Subjects Committee of the Harvard School of Public Health, as well as by the ethics approval body at each of the participating hospitals.
Provenance and peer review Not commissioned; externally peer reviewed.