You are here

Article Text

Article menu
Original research
‘Speaking up’ about patient safety concerns and unprofessional behaviour among residents: validation of two scales
  1. William Martinez1,
  2. Jason M Etchegaray2,
  3. Eric J Thomas3,
  4. Gerald B Hickson4,
  5. Lisa Soleymani Lehmann5,
  6. Anneliese M Schleyer6,
  7. Jennifer A Best6,
  8. Julia T Shelburne7,
  9. Natalie B May8,
  10. Sigall K Bell9
  1. 1Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, USA
  2. 2The RAND Corporation, Santa Monica, California, USA
  3. 3Memorial Hermann Center for Healthcare Quality and Safety, The University of Texas Medical School at Houston, Houston, Texas, USA
  4. 4Quality, Safety & Risk Prevention, Vanderbilt University Medical Center, Nashville, Tennessee, USA
  5. 5Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA
  6. 6Department of Medicine, University of Washington School of Medicine, Seattle, Washington, USA
  7. 7Department of Pediatrics, University of Texas Medical School at Houston, Houston, Texas, USA
  8. 8Department of Medicine, University of Virginia School of Medicine, Charlottesville, Virginia, USA
  9. 9Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
  1. Correspondence to Dr William Martinez, Division of General Internal Medicine, Vanderbilt University Medical Center, Nashville, TN 37232-8300, USA; william.martinez{at}vanderbilt.edu

Abstract

Objective To develop and test the psychometric properties of two new survey scales aiming to measure the extent to which the clinical environment supports speaking up about (a) patient safety concerns and (b) unprofessional behaviour.

Method Residents from six large US academic medical centres completed an anonymous, electronic survey containing questions regarding safety culture and speaking up about safety and professionalism concerns.

Results Confirmatory factor analysis supported two separate, one-factor speaking up climates (SUCs) among residents; one focused on patient safety concerns (SUC-Safe scale) and the other focused on unprofessional behaviour (SUC-Prof scale). Both scales had good internal consistency (Cronbach's α>0.70) and were unique from validated safety and teamwork climate measures (r<0.85 for all correlations), a measure of discriminant validity. The SUC-Safe and SUC-Prof scales were associated with participants’ self-reported speaking up behaviour about safety and professionalism concerns (r=0.21, p<0.001 and r=0.22, p<0.001, respectively), a measure of concurrent validity, while teamwork and safety climate scales were not.

Conclusions We created and provided evidence for the reliability and validity of two measures (SUC-Safe and SUC-Prof scales) associated with self-reported speaking up behaviour among residents. These two scales may fill an existing gap in residency and safety culture assessments by measuring the openness of communication about safety and professionalism concerns, two important aspects of safety culture that are under-represented in existing metrics.

  • Communication
  • Graduate medical education
  • Patient safety
  • Safety culture
  • Statistics

https://doi.org/10.1136/bmjqs-2015-004253

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    The 1999 Institute of Medicine Report, To Err is Human, raised awareness about medical errors in the USA and sparked a strong healthcare focus on improving organisational safety culture.1 Safety culture refers to the shared patient safety-related beliefs and behaviours of individuals within the work environment.2 Several validated questionnaires attempt to assess organisational safety climate (ie, the measurable components of safety culture).3 ,4 However, data demonstrating an association between safety culture assessments and clinician behaviour are limited.2 ,5 ,6

    Open communication regarding concerns, also called ‘speaking up’, is an essential aspect of safety culture that is receiving increased attention,7–11 yet remains under-represented in current patient safety culture questionnaires.3 ,4 Clinical situations calling for speaking up are common and include traditional patient safety concerns (eg, improper sterile technique, poor hand hygiene, inadequate pass off) and professionalism-related safety concerns (eg, covering up an error, false documentation, disruptive behaviour). Health care professionals are tasked with speaking up about concerns related to their own team's care, and about situations in which they are strictly ‘bystanders’ to safety or professionalism lapses involving other clinicians. Despite implications for patient safety, clinician silence or ineffective communication in these situations is common.12–15

    For residents, low on the medical hierarchy, speaking up when they observe threats to patient safety, including unprofessional behaviour, may be particularly challenging.16 ,17 A strong desire to ‘fit in with the team’ and fear of repercussions can trump the moral courage required to speak up about safety concerns and unprofessional behaviour.8 While educators focus on formal resident patient safety curricula,18 the hidden curriculum may be a more powerful ‘teacher’, suppressing speaking up, and shaping residents’ attitudes and behaviours.16 ,19 Recently, the Accreditation Council for Graduate Medical Education (ACGME) established the Clinical Learning Environment Review (CLER) programme to assess the role of quality and safety in residents’ learning environments, including a focus on professionalism concerns and ‘opportunities for residents to report errors, unsafe conditions, and near misses’.20 New measures will be needed to help programmes assess how existing safety and professionalism practices affect residents’ attitudes and behaviours and to chart progress in these areas over time.

    Speaking up in the context of patient safety concerns and unprofessional behaviour has well-established links to patient safety outcomes.1 ,9 ,21 However, factors affecting speaking up in each instance may be different. Lessons from the aviation industry suggest that speaking up behaviour is determined by subjective beliefs about when, what and to whom it is appropriate to speak up (ie, implicit voice theories).22 These beliefs are influenced by group norms and may be substantially different in different contexts.22 In contrast to speaking up about traditional patient safety concerns, speaking up about unprofessional behaviour is less ‘normalised’ and its implications for patient safety may be less explicit.8 ,23 ,24 Speaking up about unprofessional behaviour may be viewed as more confrontational, less acceptable and more risky, with greater potential to jeopardise existing professional relationships.25–27 While speaking up about safety concerns has been incorporated into safety curricula disseminating across the country and is advocated by safety champions,28 speaking up about unprofessional behaviour remains largely untaught. The speaking up climates (SUCs) for safety and for professionalism may therefore be different within the same clinical setting.

    We surveyed residents to develop and test the psychometric properties of two new scales aiming to measure the extent to which the clinical environment supports speaking up about (a) patient safety concerns and (b) unprofessional behaviour (ie, SUCs for safety and professionalism). Because of the differences in cultural norms, training exposure, explicit connection to safety and the personal stakes involved, we hypothesised that the survey items focused on the SUC for professionalism would be related to but distinct from those focused on the SUC for safety.

    Methods

    Participants

    All residents in internal medicine, general surgery, orthopaedic surgery, plastic surgery, urology, neurosurgery and obstetrics and gynaecology from six large academic medical centres from different geographic regions of the USA were eligible to participate. Participation in the study was voluntary and consent was implied by survey completion. All eligible residents received a gift card of small monetary value as an incentive to participate.

    Data collection

    Residents were surveyed via an email link to an electronic, anonymous questionnaire using REDCap (Research Electronic Data Capture) V.5.0.8 (Vanderbilt University, Nashville, Tennessee, USA).29 Three reminder emails were sent to non-responders every 2 weeks.

    Measures

    The survey contained questions about the SUCs for safety and professionalism, safety climate,3 teamwork climate,3 self-reported speaking up behaviour and demographics. Four physicians (WM, EJT, LSL and SKB) and one psychometrician (JME) with expertise in patient safety and scale development created two sets of survey items focusing on the extent to which speaking up is endorsed and supported within the respondent's clinical area. One set of items focused on speaking up about patient safety concerns and the other set of items focused on speaking up about unprofessional behaviour. We defined ‘clinical area’ as the respondent's overall inpatient residency training environment. We defined ‘speaking up’ as stating concerns (eg, filing a report, sharing concerns with a supervisor or speaking directly with the individual(s) involved) rather than saying nothing. When developing the two sets of items we began with language from an existing measure of patient safety culture3 and then added items to broaden our understanding of the SUC. Two items in both sets were adapted from the Safety Attitudes Questionnaire (SAQ) developed by one of the physicians (EJT).3 We built on the prior experience of three of the authors (JME, EJT and SKB) validating a perceptions of patient safety and professionalism scale for medical students to develop the speaking up about unprofessional behaviour items.25 A subset of items focused on the support for speaking up when not directly involved in the events or the patients’ care (ie, as a bystander). We revised items through pilot testing for clarity and face validity with 10 recent residency graduates.

    We included the safety and teamwork climate domains of the SAQ—Short Form in the survey to compare and contrast our new scales measuring the SUCs for safety and professionalism with existing measures of patient safety climates.3 In order to examine the concurrent validity of our scales, respondents were asked how many times during their most recent inpatient month they observed (a) a patient safety breach and (b) unprofessional behaviour. Respondents who reported observing a patient safety breach or unprofessional behaviour at least once were then asked whether they discussed the unprofessional behaviour or patient safety breach they observed with the person(s) involved (ie, self-reported speaking up behaviour). Adapted from the American College of Physicians definition,30 we defined ‘unprofessional behaviour’ as conduct of a health professional that demonstrates disrespect or lack of compassion, commitment to ethical principles, integrity or accountability toward patients or coworkers. We defined ‘patient safety breach’ as an act or omission that unnecessarily increases the risk of accidental or preventable injuries produced by medical care. Definitions included examples of patient safety breaches and unprofessional behaviour derived from the literature.31–33

    Statistical analysis

    Because we hypothesised that speaking up about patient safety and professionalism concerns represent two unique constructs, we conducted confirmatory factor analysis (CFA), rather than exploratory factor analysis, to evaluate whether our speaking up items measured their intended constructs: (1) SUC for safety and (2) SUC for professionalism. CFA is the most appropriate methodology for testing the validity with which a set of items measures its underlying construct(s).34

    To allow for cross-validation of CFA models, we partitioned the data into two groups: (1) a calibration sample composed of data from the first two institutions surveyed between March and June of 2013 and (2) a validation sample from four additional institutions surveyed between March and June of 2014. We used descriptive statistics to describe the two samples and χ2 test to analyse differences in proportions between the two groups.

    Because no consensus exists regarding the best model fit indices,35–37 we examined several common indices to assess the fit of our CFA models including χ2/degrees of freedom ratio (χ2/df ratio), root mean square error approximation (RMSEA), standardised root mean square residual (SRMR), comparative fit index (CFI) and normed fit index (NFI). We considered good model fit to include χ2/df ratio<3, RMSEA <0.06, SRMR <0.08, CFI values ≥0.95 and NFI ≥0.95, and RMSEA values between 0.06 and 0.10 to suggest moderate fit.34–36 We assessed the internal consistency of each factor with Cronbach's α, and considered a Cronbach's α of ≥0.70 as suggesting good internal consistency.38 To determine which survey items effectively measured each of the SUCs (ie, safety (Safe) and professionalism (Prof)), we computed an initial CFA model separately for each set of items (SUC-Safe and SUC-Prof) on the calibration sample (institutions A and B). Because several survey items in each set (SUC-Safe and SUC-Prof) were related to speaking up as a bystander, we covaried the error terms of these items within each model.39 ,40 Initial models that did not satisfy all five models fit indices were iteratively revised by deleting items based on regression weights and item content to determine the best model fit. The final revised models developed using the calibration sample were then tested on the validation sample (institutions C–F) to determine if model fit was consistent across both samples.

    All scale items were measured on a 5-point Likert scale from ‘strongly disagree’ (1) to ‘strongly agree’ (5). Negatively worded items were reverse coded, so that higher scores represent more positive perceptions for all scale items. A standardised summary score was computed for all scales using the following formula: scale score=(average item score—1)×25. Thus, summary scores for each scale could range from 0 (worst) to 100 (best). SUC-Safe and SUC-Prof scale scores were created using the reduced item set that came out of the CFA. We used multivariate linear regression to model sociodemographic predictors of scale scores.

    For each institution, we calculated the ‘per cent-positive’, the proportion of participants with positive perceptions (ie, a scale score ≥75), on each climate scale. Organisations typically interpret per cent-positive of ≥80% on teamwork and safety climate scales as ideal performance, and per cent-positive of <60% as elements of patient safety culture that need improvement.41 We used McNemar's test to measure differences in the proportions of participants with positive perceptions on different climate scales within the same institution.

    To examine the discriminant validity of the scales for each of the SUCs, we computed the correlations between validated safety climate and teamwork climate scales3 and scales for each of the SUCs (Safe and Prof). Correlations between scales <0.85 indicate discriminant validity.37 ,42 Given the similarity in the wording of two items in the SUC-Safe to an item each in the SAQ teamwork and safety domains, we measured the interitem correlations between all scale items and further examined discriminant validity using a pairwise, nested model approach described by Vogus and Sutcliffe.39

    To estimate the concurrent validity of the scales for each of the SUCs compared with existing safety climate and teamwork climate scales, we measured the correlations between each of the climate scales and the self-reported speaking up behaviour of participants. For this analysis, participants who reported observing a patient safety breach or unprofessional behaviour, respectively, were dichotomised into two groups: participants that reported speaking up at least once about what they observed versus those who did not speak up at all. In addition, we used t tests to compare the mean SUC-Safe and SUC-Prof scale scores between the two groups. To account for multiple comparisons, we applied the Bonferroni correction when interpreting the significance of the correlational results, comparisons of per cent-positive scores on climate scales within institutions and the results of the multivariate regression analysis.

    Analyses were performed using SAS V.9.4 (SAS Institute, Cary, North Carolina, USA) and IBM SPSS Amos V.22.0.0 (Amos Development, Meadville, Pennsylvania, USA). The study was approved by each site's institutional review board.

    Results

    Surveys were completed by 47% (837/1800) of eligible residents. Table 1 describes respondents’ characteristics. The calibrations and validation samples contained nearly equal proportions of respondents by gender, specialty and postgraduate year. The validation sample contained a significantly greater proportion of respondents who reported receiving formal training in patient safety than the calibration sample (457/484 (95%) versus 291/353 (82%), χ2=25.49, p<0.001).

    View this table:
    Table 1

    Characteristics of 837 respondents to an anonymous survey of speaking up climates for safety and professionalism, safety climate and teamwork climate*

    Table 2 reports the results of the CFA on the two sets of survey items focusing on the SUCs for safety and professionalism, respectively. The initial CFA for the 6-item, one-factor SUC for safety model demonstrated good fit in the calibration sample on four of the five model fit indices (χ2/df=2.77, SRMR=0.04, CFI=0.98, NFI=0.97) and moderate model fit on the remaining index (RMSEA=0.07) (see table 3 for items within each factor). Of the six items, item 1 had the lowest regression weight. Unlike items 2 through 6, which assessed respondents' perceptions of their clinical area, item 1 assessed respondents’ personal attitudes regarding speaking up about patient safety concerns (ie, the respondent, not the clinical area, is the referent for the item). After deleting item 1, model fit indices improved and suggested good fit across all five indices (χ2/df=1.41, RMSEA=0.03, SRMR=0.02, CFI=1.00 and NFI=0.99). Cronbach's α for this 5-item factor was 0.79. We labelled this factor the Speaking Up Climate for Patient Safety (SUC-Safe) Scale.

    View this table:
    Table 2

    Results of initial and revised confirmatory factor analysis models for speaking up climates

    View this table:
    Table 3

    Mean Likert scale score for survey items of the SUC-Safe and SUC-Prof scales, and the SAQ Short Form Scale teamwork and safety domains

    The initial CFA for the 7-item, one-factor SUC for professionalism model initially demonstrated poor fit in the calibration sample on all five model fit indices (χ2/df=7.06, RMSEA=0.13, SRMR=0.07, CFI=0.89 and NFI=0.87). The regression weights for items 1, 2 and 4 were lower than the other items. Similar to the items deleted from the initial model of SUC-Safe scale, the referent for items 1 and 2 was the respondent rather than the clinical area. Although item 4 pertains to the clinical area, its referent is also the respondent—assessing individual comfort speaking up, which is influenced by respondents’ characteristics (eg, assertiveness) in addition to climate. It is also possible that item 4’s lower regression weight may be due to a measurement artefact related to its negative wording relative to the remaining items.43 ,44 After iteratively revising the model by deleting items 1, 2 and 4 in succession, model fit indices improved and suggested good fit across all five fit indices (χ2/df=1.60, RMSEA=0.04, SRMR=0.01, CFI=1.00 and NFI=1.00). Cronbach's α for this 4-item factor was 0.75. We labelled this factor the Speaking Up Climate for Professionalism (SUC-Prof) Scale.

    CFA of the revised SUC-Safe model on the validation sample demonstrated good fit, comparable to the model fit demonstrated on the calibration sample, on three of the five fit indices (SRMR=0.02, CFI=0.98 and NFI=0.98) and moderate fit on two of the five indices (χ2/df=3.82, RMSEA=0.08). CFA of the revised SUC-Prof model on the validation sample demonstrated good fit, comparable to the model fit demonstrated on the calibration sample, across all five fit indices (χ2/df=0.52, RMSEA=0.00, SRMR=0.01, CFI=1.00 and NFI=1.00).

    Table 3 contains the mean Likert scale scores for the items within the SUC-Safe and SUC-Prof scales, and the items within the SAQ Short Form Scale Teamwork and Safety domains. The two lowest mean Likert scale scores within both the SUC-Safe and SUC-Prof scales belonged to items related to speaking up as a bystander (see SUC-Safe items 4 and 5 and SUC-Prof items 5 and 6). The lowest mean Likert scale score within the SAQ Short Form Scale—Teamwork domain belonged to an item assessing speaking up when a problem with patient care is perceived (see SAQ Short Form Scale—Teamwork domain, item 2). The lowest mean Likert scale score within the SAQ Short Form Scale—Safety domain belonged to an item related to the discussion of errors (see SAQ Short Form Scale—Safety domain, item 5). Overall, mean scores for items in the SAQ Teamwork and Safety domains were higher than those for SUC-Safe and SUC-Prof.

    Table 4 shows the proportion of respondents within each institution with positive perceptions for each patient safety-related climate scale. The proportion of respondents with positive perceptions of the SUC-Prof was significantly lower than the SAQ-Teamwork climate and SAQ-Safety climate in all institutions surveyed. It was also lower than the SUC-Safe in all but institution E (p<0.001 for all significant differences in proportions). In addition, the proportion of respondents with positive perceptions of the SUC-Safe was significantly lower than SAQ-Teamwork climate and SAQ-Safety in most institutions (table 4, p<0.001 for all significant differences in proportions).

    View this table:
    Table 4

    Per cent-positive scores for patient safety climate scales by institution

    Table 5 contains the bivariate correlations among the SAQ-Teamwork, SAQ-Safety, SUC-Prof and SUC-Safe scales and participants’ self-reported speaking up behaviour about patient safety breaches and unprofessional behaviour. Correlations between the scales were all <0.85, indicating that the scales for each of the SUCs measured unique domains of safety culture as compared with existing safety climate and teamwork climate scales.37 ,42 Among residents who reported observing at least one patient safety breach during their most recent inpatient month, both the SUC-Safe scale and the SUC-Prof scale were significantly associated with self-reported speaking up behaviour about the breach(es) (r=0.21, p<0.001 and r=0.22, p<0.001; respectively). Among residents who reported observing unprofessional behaviour at least once during their most recent inpatient month, only the SUC-Prof scale was significantly associated with self-reported speaking up behaviour about the behaviour(s) (r=0.22, p<0.001). In both instances, teamwork and safety climate scales were not associated with self-reported speaking up behaviour. In addition, online supplementary table S6 shows the SUC-Safe and SUC-Prof scales’ means by respondents’ self-reported speaking up behaviour. The SUC-Safe scale mean was higher among residents that reported observing and speaking up about patient safety breach(es) than those that reported observing and remaining silent (M=3.52 vs M=3.22, p<0.001). Similarly, the mean SUC-Prof scale score was higher among residents that reported observing and speaking up about unprofessional behaviour(s) than those that reported observing and remaining silent (M=3.12 vs M=2.76, p<0.001).

    View this table:
    Table 5

    Correlations among patient safety climate scales and self-reported speaking up behaviour

    Items 3 and 6 in the SUC-Safe scale were similar to item 2 of the SAQ teamwork domain and item 6 in the SAQ safety domain (see table 3), respectively; thus we also analysed discriminant validity using a CFA approach (see online supplementary table S7) outlined by Vogus and Sutcliffe.39 Online supplementary table S8 shows the interitem correlations of all scale items. The correlation between item 3 of the SUC-Safe scale and item 2 of the SAQ teamwork domain was 0.52 (p<0.001), and the correlation between item 6 of the SUC-Safe scale and item 6 of the SAQ safety domain was 0.61 (p<0.001). These collective findings further supported that SUC-Safe and SUC-Prof are distinct from SAQ teamwork and safety domains.

    Multivariate linear regression analysis of sociodemographic predictors of the SUC-Safe and SUC-Prof scale scores showed that SUC-Safe scores were positively associated with receiving prior formal patient safety training (B=11.75, p<0.001), while SUC-Prof scores were not. The latter scores were positively associated with male gender (B=6.07, p<0.001) (see online supplementary table S9).

    Discussion

    In this study, we provide preliminary evidence for the reliability and validity of two new scales measuring the SUCs for safety (SUC-Safe) and professionalism (SUC-Prof) among residents. To our knowledge, our study is the first to attempt to measure the SUCs related to both traditional patient safety concerns and unprofessional behaviour and to determine the extent to which they relate to self-reported speaking up behaviour.

    Our results highlight several key findings. First, we found that the scales for the SUCs for both safety and professionalism were substantially different from each other and from two commonly used and validated measures of safety and teamwork climate. Second, SUC-Safe and SUC-Prof were each associated with participants’ self-reported speaking up behaviour about patient safety breaches and unprofessional conduct while safety and teamwork climate were not, suggesting that existing measures may not adequately assess this aspect of safety culture. Finally, we also found that the SUCs for professionalism and safety concerns were worse than the safety and teamwork climates as perceived by the residents at each of the six academic institutions we surveyed, with only a third or fewer residents endorsing positive SUC for safety and less than a fifth of residents endorsing a positive SUC for professionalism concerns. Mean scores on the items addressing speaking up as a bystander for both safety and professionalism were particularly low.

    Our findings have several implications. SUCs for safety and professionalism may be poor even when safety and teamwork climates are relatively good. As institutions work to improve safety, such deficits may be barriers to progress and highlight the need for special attention to promote speaking up about safety and professionalism concerns, and particularly among bystanders. Educational and organisational leaders might incorporate advanced communication skills for speaking up about safety and professionalism concerns into existing patient safety curricula and initiate curricular efforts that highlight collective accountability for the clinical learning environment, the importance of speaking up and the link between unprofessional behaviour and patient safety.8 ,25 ,32 Ensuring appropriate role modelling for learners and reducing medical hierarchy in favour of teamwork and collective accountability for safety will also be critical to promoting speaking up.16 ,19 In addition, mean scores for questions assessing resident perceptions of meaningful institutional change following speaking up about unprofessional behaviour were also notably low. Thus, educational and organisational leaders need to consider reliable non-punitive systems for collecting and tracking concerns about unprofessional behaviour and transparent ways to hold professionals accountable for their actions and to ensure that the reports of those who speak up are taken seriously.32 ,45 Overall, our results suggest that measuring the SUCs for safety and professionalism can reveal important safety gaps as well as actionable information for residency programmes and healthcare organisations.

    Prior studies found limitations of the SAQ subscales (eg, safety and teamwork climate), including that they do not predict the number of safety events reported within the preceding 12 months or clinicians’ intent to disclose medical errors.2 ,46 Similarly, we found that the safety and teamwork dimensions of the SAQ did not predict self-reported speaking up behaviour among residents. Although some broad patient safety climate instruments possess limited items about speaking up, they only capture a portion of the elements important in assessing the SUC for safety, and generally even fewer for the SUC for professionalism. Thus, residency programmes and healthcare organisations should consider adding survey items about the SUCs for safety and professionalism to annual patient safety culture assessments and residency programme evaluations. The SUC-Safe and SUC-Prof scales may help residency programmes and healthcare organisations assess the extent to which their culture encourages speaking up about safety and professionalism concerns. Such assessments may help uncover an otherwise unmeasured barrier to safety in the clinical learning environment, and one to which medical students, residents, fellows and nurses may be particularly vulnerable because of medical hierarchy effects. Regularly assessing these SUCs will allow changes to be tracked over time and may assist residency programmes in meeting new ACGME CLER standards,20 by more comprehensively assessing the quality and safety of the clinical learning environment. Explicitly measuring SUCs can send a clear message about institutional priorities, and may in and of itself influence behaviour.47

    While residents face particular challenges with speaking up, they are not alone.11 ,48 Our scales may assist in assessing perceptions of the SUC among other healthcare professionals including students, nurses, attending physicians and others. Future studies should examine the psychometric properties of the SUC-Safe and SUC-Prof scales among other healthcare professionals and in other settings such as outpatient clinics and community hospitals. Additional items specific to these groups and settings may allow a more nuanced assessment of the SUCs for safety and professionalism within different clinical areas and help target organisational improvement goals and strategies. Future studies might also explore other SUCs such as speaking up about knowledge deficits and personal limitations among learners. In addition, more research is needed to identify the environmental and individual factors that facilitate or impede speaking up about safety and professionalism concerns, and to link SUCs to safety and quality outcomes. Finally, more research is needed to assess whether specific interventions can improve attitudes and behaviour regarding speaking up, and explore the relationship between SUCs and the actual content and quality of communication between professionals when raising concerns. Investigators exploring such issues or adopting the SUC-Safe and SUC-Prof in their own settings may note that the items deleted from the initial SUC-Safe and SUC-Prof models (based on their psychometric properties) may still provide important information about respondent attitudes related to speaking up. However, they should not be used when calculating climate scores.

    Finally, our study produced a few tentative, yet important, findings that merit further attention. First, we found that reporting prior formal training in patient safety was associated with more positive perceptions of the SUC for safety but not for professionalism. This illustrates the known difficulty physicians have in raising professionalism concerns and highlights a gap in current patient safety curricula.14 ,49 Second, we found that women residents had more negative perceptions of the SUC for professionalism than men. These findings may be due to gender-based differences of empowerment and confidence consistent with prior research.19 ,50 More research is needed to confirm, better understand and remedy these differences.

    Our study has several important limitations. First, although consistent with other studies of physicians,51 ,52 our response rate was moderate and non-response bias may affect the generalisability of the results. However, our study was primarily focused on assessing the internal validity of the SUC scales as opposed to the generalisability of results. Thus, the data were adequate for this purpose. Second, residents were asked to recall past events making the study subject to recall bias. However, experiences with patient safety breaches and unprofessional behaviour may be salient events and thus may be recalled more accurately than more routine events. Third, although the survey was anonymous, it relied on self-reported attitudes and behaviours and thus may have been subject to social desirability bias. If social desirability bias was present, it would mean that our relatively low scale scores for both SUCs are overestimates and the reality may be even more concerning than our data suggest. Finally, while the model fit for SUC-Safe scale in both samples was good as assessed by three fit indices (SRMR, CFI and NFI), two other indices (χ2/df and RMSEA) suggested good fit in the calibration sample and moderate fit in the validation sample. This may be due to variability in the data, overfitting of the SUC-Safe model in the calibration sample or to the low degrees of freedom in our final models. Both χ2/df and RMSEA have been criticised for bias favouring models with higher degrees of freedom.53 ,54 For this reason, some have argued not to compute the RMSEA for low degrees of freedom models and proposed a cut-off of 5 rather than 3 for χ2/df ratio.53 ,54 To reduce variability, future studies may employ multiple rounds of cross-validation performed on different partitions of the study sample with the validation results averaged over the rounds.

    In conclusion, ensuring openness of communication between healthcare professionals regarding traditional patient safety concerns and professionalism-related safety concerns is an essential aspect of safety culture, but may not be adequately measured by existing metrics. Thus, developing valid methods for assessing the SUC is critical. We report the first survey that may reliably and validly measure the SUC for safety and professionalism concerns among residents. The striking gaps in residents’ perceptions of support for speaking up as either a team member or a bystander suggest unaddressed needs and ongoing safety deficits in the clinical learning environment. These scales may be used as stand-alone measures, or as an add-on to existing safety culture or residency programme assessments. They can help programmes and healthcare organisations assess SUCs, identify educational and organisational needs and track their progress in response to targeted curricular interventions.

    Acknowledgments

    The authors wish to thank the residents who participated in the study and their residency program directors who supported this project. Dr Bell thanks the Arnold P. Gold Foundation for a career development award through a Gold Professorship.

    References

    1. ↵
      Institute of Medicine. To err is human: building a safer health system. Washington DC: National Academy Press, 1999.
    2. ↵
      1. Etchegaray JM,
      2. Thomas EJ
      . Comparing two safety culture surveys: safety attitudes questionnaire and hospital survey on patient safety. BMJ Qual Saf 2012;21:490–8. doi:10.1136/bmjqs-2011-000449
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. Sexton JB,
      2. Helmreich RL,
      3. Neilands TB, et al
      . The safety attitudes questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 2006;6:44. doi:10.1186/1472-6963-6-44
      OpenUrlCrossRefPubMed
    4. ↵
      1. Sorra JS,
      2. Dyer N
      . Multilevel psychometric properties of the AHRQ hospital survey on patient safety culture. BMC Health Serv Res 2010;10:199. doi:10.1186/1472-6963-10-199
      OpenUrlCrossRefPubMed
    5. ↵
      1. Colla JB,
      2. Bracken AC,
      3. Kinney LM, et al
      . Measuring patient safety climate: a review of surveys. Qual Saf Health Care 2005;14:364–6. doi:10.1136/qshc.2005.014217
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Weaver SJ,
      2. Lubomksi LH,
      3. Wilson RF, et al
      . Promoting a culture of safety as a patient safety strategy: a systematic review. Ann Intern Med 2013;158(5 Pt 2):369–74. doi:10.7326/0003-4819-158-5-201303051-00002
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      1. Beyea SC
      . Speaking up for quality and safety. AORN J 2008;88:115–16. doi:10.1016/j.aorn.2008.06.010
      OpenUrlCrossRefPubMed
    8. ↵
      1. Liao JM,
      2. Thomas EJ,
      3. Bell SK
      . Speaking up about the dangers of the hidden curriculum. Health Aff (Millwood) 2014;33:168–71. doi:10.1377/hlthaff.2013.1073
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Okuyama A,
      2. Wagner C,
      3. Bijnen B
      . Speaking up for patient safety by hospital-based health care professionals: a literature review. BMC Health Serv Res 2014;14:61. doi:10.1186/1472-6963-14-61
      OpenUrlCrossRefPubMed
    10. ↵
      1. Sayre MM,
      2. McNeese-Smith D,
      3. Leach LS, et al
      . An educational intervention to increase “speaking-up” behaviors in nurses and improve patient safety. J Nurs Care Qual 2012;27:154–60. doi:10.1097/NCQ.0b013e318241d9ff
      OpenUrlCrossRefPubMed
    11. ↵
      1. Lyndon A,
      2. Sexton JB,
      3. Simpson KR, et al
      . Predictors of likelihood of speaking up about safety concerns in labour and delivery. BMJ Qual Saf 2012;21:791–9. doi:10.1136/bmjqs-2010-050211
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Dankoski ME,
      2. Bickel J,
      3. Gusic ME
      . Discussing the undiscussable with the powerful: why and how faculty must learn to counteract organizational silence. Acad Med 2014;89:1610–13. doi:10.1097/ACM.0000000000000428
      OpenUrlCrossRefPubMed
    13. ↵
      1. Maxfield D,
      2. Grenny J,
      3. McMillan R, et al
      . Silence kills—the seven crucial conversations for healthcare. VitalSmarts, LC, 2006. http://wwwsilencekillscom/.
    14. ↵
      1. DesRoches CM,
      2. Rao SR,
      3. Fromson JA, et al
      . Physicians’ perceptions, preparedness for reporting, and experiences related to impaired and incompetent colleagues. JAMA 2010;304:187–93. doi:10.1001/jama.2010.921
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      1. Gallagher TH,
      2. Mello MM,
      3. Levinson W, et al
      . Talking with patients about other clinicians’ errors. NEJM 2013;369:1752–7. doi:10.1056/NEJMsb1303119
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Martinez W,
      2. Lehmann LS
      . The “hidden curriculum” and residents’ attitudes about medical error disclosure: comparison of surgical and nonsurgical residents. J Am Coll Surg 2013;217:1145–50. doi:10.1016/j.jamcollsurg.2013.07.391
      OpenUrlCrossRefPubMed
    17. ↵
      1. Cosby KS,
      2. Croskerry P
      . Profiles in patient safety: authority gradients in medical error. Acad Emerg Med 2004;11:1341–5. doi:10.1111/j.1553-2712.2004.tb01925.x
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Wong BM,
      2. Etchells EE,
      3. Kuper A, et al
      . Teaching quality improvement and patient safety to trainees: a systematic review. Acad Med 2010;85:1425–39. doi:10.1097/ACM.0b013e3181e2d0c6
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      1. Martinez W,
      2. Hickson GB,
      3. Miller BM, et al
      . Role-modeling and medical error disclosure: a national survey of trainees. Acad Med 2014;89:482–9. doi:10.1097/ACM.0000000000000156
      OpenUrlCrossRefPubMed
    20. ↵
      1. Weiss KB,
      2. Wagner R,
      3. Nasca TJ
      . Development, testing, and implementation of the ACGME Clinical Learning Environment Review (CLER) program. J Grad Med Educ 2012;4:396–8. doi:10.4300/JGME-04-03-31
      OpenUrlCrossRefPubMed
    21. ↵
      1. Pichert JW,
      2. Moore IN,
      3. Hickson GB
      . Professionals promoting professionalism. Jt Comm J Qual Patient Saf 2011;37:446.
      OpenUrlPubMed
    22. ↵
      1. Bienefeld N,
      2. Grote G
      . Silence that may kill: when aircrew members don't speak up and why. Aviat Psychol Appl Hum Factors 2012;2:1–10. doi:10.1027/2192-0923/a000021
      OpenUrlCrossRef
    23. ↵
      1. Landon BE,
      2. Normand SLT,
      3. Blumenthal D, et al
      . Physician clinical performance assessment: prospects and barriers. JAMA 2003;290:1183–9. doi:10.1001/jama.290.9.1183
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Leape LL,
      2. Fromson JA
      . Problem doctors: is there a system level solution? Ann Intern Med 2006;144:107–16. doi:10.7326/0003-4819-144-2-200601170-00008
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Liao JM,
      2. Etchegaray JM,
      3. Williams ST, et al
      . Assessing medical students’ perceptions of patient safety: the medical student safety attitudes and professionalism survey. Acad Med 2014;89:343–51. doi:10.1097/ACM.0000000000000124
      OpenUrlCrossRefPubMed
    26. ↵
      Joint Commission. Behaviors that undermine a culture of safety. Sentinel Event Alert 2008;40:1–5.
      OpenUrlPubMed
    27. ↵
      1. Rosenstein AH
      . Original research: nurse-physician relationships: impact on nurse satisfaction and retention. Am J Nurs 2002;102:26–34. doi:10.1097/00000446-200206000-00040
      OpenUrlPubMed
    28. ↵
      1. Clancy CM,
      2. Tornberg DN
      . TeamSTEPPS: assuring optimal teamwork in clinical settings. Am J Med Qual 2007;22:214–17. doi:10.1177/1062860607300616
      OpenUrlFREE Full Text
    29. ↵
      1. Harris PA,
      2. Taylor R,
      3. Thielke R, et al
      . Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377–81. doi:10.1016/j.jbi.2008.08.010
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      ABIM Foundation. American Board of Internal Medicine. ACP-ASIM Foundation. American College of Physicians-American Society of Internal Medicine; European Federation of Internal Medicine. Medical professionalism in the new millennium: a physician charter. Ann Intern Med 2002;136:243–6. doi:10.7326/0003-4819-136-3-200202050-00012
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      1. Reddy ST,
      2. Iwaz JA,
      3. Didwania AK, et al
      . Participation in unprofessional behaviors among hospitalists: a multicenter study. J Hosp Med 2012;7:543–50. doi:10.1002/jhm.1946
      OpenUrlCrossRefPubMed
    32. ↵
      1. Hickson GB,
      2. Pichert JW,
      3. Webb LE, et al
      . A complementary approach to promoting professionalism: identifying, measuring, and addressing unprofessional behaviors. Acad Med 2007;82:1040–8. doi:10.1097/ACM.0b013e31815761ee
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      1. Makeham MA,
      2. Stromer S,
      3. Bridges-Webb C, et al
      . Patient safety events reported in general practice: a taxonomy. Qual Saf Health Care 2008;17:53–7. doi:10.1136/qshc.2007.022491
      OpenUrlAbstract/FREE Full Text
    34. ↵
      1. Bryant FB,
      2. Yarnold PR
      . Principal-components analysis and exploratory and confirmatory factor analysis. In: Yarnold LGGPR, ed. Reading and understanding multivariate statistics. Washington DC, USA: American Psychological Association, 1995:99–136.
    35. ↵
      1. Hu L,
      2. Bentler PM
      . Cutoff criteria for fit indexes in covariance structural analysis: conventional criteria versus new alternatives. Struct Equation Model 1999;6:1–55. doi:10.1080/10705519909540118
      OpenUrlCrossRef
    36. ↵
      1. Bollen KA
      . Structural equations with latent variables. New York, NY: Wiley, 1989.
    37. ↵
      1. Kline RB
      . Principles and practice of structural equation modeling. New York, NY: The Guilford Press, 2005.
    38. ↵
      1. Terwee CB,
      2. Bot SD,
      3. de Boer MR, et al
      . Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34–42. doi:10.1016/j.jclinepi.2006.03.012
      OpenUrlCrossRefPubMedWeb of Science
    39. ↵
      1. Vogus TJ,
      2. Sutcliffe KM
      . The safety organizing scale: development and validation of a behavioral measure of safety culture in hospital nursing units. Med Care 2007;45:46–54. doi:10.1097/01.mlr.0000244635.61178.7a
      OpenUrlCrossRefPubMedWeb of Science
    40. ↵
      1. Mueller RO,
      2. Hancock GR
      . Best practices in structural equation modeling. Best Practice Quant Methods 2008:488–508. doi:10.4135/9781412995627.d38
    41. ↵
      1. Timmel J,
      2. Kent PS,
      3. Holzmueller CG, et al
      . Impact of the Comprehensive Unit-based Safety Program (CUSP) on safety culture in a surgical inpatient unit. Jt Comm J Qual Patient Saf 2010;36:252–60.
      OpenUrlPubMed
    42. ↵
      1. Nunnally JC,
      2. Bernstein IH
      . Psychometric theory. 3rd edn. New York, NY: McGraw-Hill, 1994.
    43. ↵
      1. Schriesheim CA,
      2. Hill KD
      . Controlling acquiescence response bias by item reversals: The effect on questionnaire validity. Educ Psychol Meas 1981;41:1101–14. doi:10.1177/001316448104100219
      OpenUrlAbstract/FREE Full Text
    44. ↵
      1. Pilotte WJ,
      2. Gable RK
      . The impact of positive and negative item stems on the validity of a computer anxiety scale. Educ Psychol Meas 1990;50:603–10. doi:10.1177/0013164490503016
      OpenUrlAbstract
    45. ↵
      1. Martinez W,
      2. Pichert JW,
      3. Hickson GB, et al
      . Programs for promoting professionalism: questions to guide next steps. Jt Comm J Qual Patient Saf 2014;40:159–60.
      OpenUrlPubMed
    46. ↵
      1. Etchegaray JM,
      2. Gallagher TH,
      3. Bell SK, et al
      . Error disclosure: a new domain for safety culture assessment. BMJ Qual Saf 2012;21:594–9. doi:10.1136/bmjqs-2011-000530
      OpenUrlAbstract/FREE Full Text
    47. ↵
      1. Singer SJ,
      2. Vogus TJ
      . Safety climate research: taking stock and looking forward. BMJ Qual Saf 2013;22:1–4. doi:10.1136/bmjqs-2012-001572
      OpenUrlFREE Full Text
    48. ↵
      1. Srivastava R
      . Speaking up—when doctors navigate medical hierarchy. NEJM 2013;368:302–5. doi:10.1056/NEJMp1212410
      OpenUrlCrossRefPubMed
    49. ↵
      1. Lucey C,
      2. Souba W
      . Perspective: the problem with the problem of professionalism. Acad Med 2010;85:1018–24. doi:10.1097/ACM.0b013e3181dbe51f
      OpenUrlCrossRefPubMedWeb of Science
    50. ↵
      1. Bartels C,
      2. Goetz S,
      3. Ward E, et al
      . Internal medicine residents’ perceived ability to direct patient care: impact of gender and experience. J Women's Health (Larchmt) 2008;17:1615–21. doi:10.1089/jwh.2008.0798
      OpenUrlCrossRefPubMed
    51. ↵
      1. Kellerman SE,
      2. Herold J
      . Physician response to surveys. A review of the literature. Am J Prev Med 2001;20:61–7. doi:10.1016/S0749-3797(00)00258-0
      OpenUrlPubMedWeb of Science
    52. ↵
      1. Singer SJ,
      2. Gaba DM,
      3. Geppert JJ, et al
      . The culture of safety: results of an organization-wide survey in 15 California hospitals. Qual Saf Health Care 2003;12:112–18. doi:10.1136/qhc.12.2.112
      OpenUrlAbstract/FREE Full Text
    53. ↵
      1. Chen F,
      2. Curran PJ,
      3. Bollen KA, et al
      . An Empirical Evaluation of the Use of Fixed Cutoff Points in RMSEA Test Statistic in Structural Equation Models. Sociol Methods Res 2008;36:462–94. doi:10.1177/0049124108314720
      OpenUrlAbstract/FREE Full Text
    54. ↵
      1. Kenny DA,
      2. Kaniskan B,
      3. McCoach DB
      . The performance of RMSEA in models with small degrees of freedom. Sociol Methods Res 2014. Published Online First 24 July 2014. doi:10.1177/0049124114543236.

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter Follow William Martinez at @DrWillMartinez

    • Contributors Study conception and design: WM, JME, EJT, GBH, LSL, SKB. Acquisition of data: WM, AMS, JAB, NBM and JTS. Analysis of data: WM and JME. Interpretation of data: WM, JME, EJT, GBH, LSL, AMS, JAB, NBM, JTS and SKB. Drafting of manuscript: WM. Critical revision: WM, JME, EJT, GBH, LSL, AMS, JAB, NBM, JTS and SKB.

    • Funding American Philosophical Society; National Center for Advancing Translational Sciences (UL1 TR000445); The Arnold P. Gold Foundation; Brigham and Women's Hospital; National Institute of Child Health and Human Development (1K24HD053771); Health Resources and Services Administration (T32HP10251).

    • Competing interests None declared.

    • Ethics approval IRB at each study site.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    Linked Articles