Content validity and internal consistency of the Dutch translation of the Safety Attitudes Questionnaire: An observational study

https://doi.org/10.1016/j.ijnurstu.2011.10.002Get rights and content

Abstract

Background

Patient safety is fundamental to healthcare quality. Attention has recently focused on the patient safety culture of an organisation and its impact on patient outcomes. A strong safety climate appears to be an essential condition for safe patient care in the hospital. A number of instruments are used to measure this patient safety climate or culture. The Safety Attitudes Questionnaire is a validated, widely used instrument to investigate multiple dimensions of safety climate at the clinical level in a variety of inpatient and outpatient settings.

Objectives

The purpose of this study is to explore the face- and content validity and the internal consistency of the Safety Attitudes Questionnaire in a large Belgian academic medical center.

Method

The translation into Dutch was done by three researchers. A panel of fifteen Dutch speaking experts evaluated the translation and its content validity. Content validity was quantified by the content validity index (CVI) and a modified kappa index. Face validity was evaluated by two nurses and two physicians who assessed the Dutch version of the SAQ. A cross-sectional design was used to test internal consistency of the SAQ items by calculating Cronbach's alpha and corrected item-total correlations.

Results

Twenty-three of the 33 SAQ items showed excellent and seven items showed good content validity. One item had a fair kappa value (item 20) and two items had a low content validity index (items 15 and 16). The average CVI of the total scale was 0.83 and ranged from 0.55 to 0.97 for the six subscales. The face-validity was good with no fundamental remarks given. The SAQ's overall Cronbach's alpha was 0.9 and changed minimally when removing items. The item-total correlations ranged from 0.10 to 0.63, no single items were strongly correlated with the sum of the other items.

Conclusion

We conclude that in this study the Dutch version of the Safety Attitudes Questionnaire showed acceptable to good psychometric properties. In line with previous evidence, this instrument seems to be an acceptable to adequate tool to evaluate the safety climate.

Introduction

Patient safety is fundamental to healthcare quality and is on the research agenda of numerous policy institutions in many countries. Attention has recently focused on the patient safety culture of an organisation and its impact on patient outcomes (Colla et al., 2005, Huang et al., 2010, Hudson et al., 2009). According to the World Health Organisation, every point in a process of care-giving contains a certain risk of inherent unsafety (WHO, 2011). Studies demonstrate that about one out of ten hospitalized patients in Western countries are affected by adverse events. This is an injury secondary to medical care and not a result of a patient's underlying medical condition (Institute of medicine, 2000). Approximately one-half of the adverse events was considered preventable (De Vries et al., 2008).

A strong safety culture is essential for in-hospital patient safety and reduces the risk of adverse events (Kline et al., 2008). Patient safety culture is defined as a “subset of organisational culture, which relates specifically to the values and beliefs concerning patient safety within healthcare organisations” (Feng et al., 2008). In essence, culture can be expressed as “the way we do things around here on our work unit” (Pronovost and Sexton, 2005). A more readily measurable aspect of safety culture at the local workplace is the ‘climate of patient safety’, and was described as “the product of individual and group values, attitudes, perceptions, competencies, and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation's health and safety management” (Health and Safety Commission, 1993). However, climate and culture are used interchangeably in the literature.

Patient safety culture/climate measures are widely used as diagnostic tools to identify areas for improvement of care processes. It can also be an onset for patient safety programs which can raise awareness about the role of an appropriate culture in promoting a safer patient environment (Nieva and Sorra, 2003, Pronovost and Sexton, 2005).

A variety of instruments exists to measure an organisation's patient safety climate such as the Safety Attitudes Questionnaire (Sexton et al., 2006a), the Hospital Survey on Patient Safety Culture (Agency for Healthcare Research and Quality, 2011) or the Safety Organising Scale (Vogus and Sutcliffe, 2007). These instruments measure domains of safety culture such as management/supervisors, safety systems, risk perception, job demands, reporting/speaking up, safety attitudes/behaviours, communication/feedback, teamwork, personal resources and organisational factors (Hellings and Vleugels, 2009). The strengths of these tools vary and the SAQ is recommended by several authors (Sexton et al., 2006a, Modak et al., 2007, Deilkas and Hofoss, 2008, Kaya et al., 2010, Nordén-Hägg et al., 2010, Poley et al., 2011) because of the following reasons. The validity and reliability of the SAQ has been documented in many countries such as the United States (Modak et al., 2007, Sexton et al., 2006a), Norway (Deilkas and Hofoss, 2008), Turkey (Kaya et al., 2010), Sweden (Nordén-Hägg et al., 2010) and the Netherlands (Poley et al., 2011). Furthermore, the instrument has also been adapted for the use in different settings and specializations such as intensive care units (ICUs) (France et al., 2010, Pronovost et al., 2008, Poley et al., 2011, Sexton et al., 2011), operating rooms (Carney et al., 2010), general inpatient settings such as medical and surgical wards (Deilkas and Hofoss, 2008), ambulatory clinics (Holden et al., 2009, Modak et al., 2007), pharmacies (Nordén-Hägg et al., 2010), and labour and delivery units (Sexton et al., 2006b). Also the SAQ has been extensively used to explore the relationship between safety climate scores and patient outcomes, e.g. medication errors, adverse events,… (Colla et al., 2005, Huang et al., 2010, Hudson et al., 2009). Finally, all versions of the SAQ have the same contents for the various items, with minor modifications to reflect the clinical area. Because of the above-mentioned reasons we have chosen to translate this instrument and validate the Belgian version of the Dutch SAQ.

The Safety Attitudes Questionnaire (SAQ) developed by Sexton et al. (2006a) is based on two conceptual models: Vincent's framework for analysing risk and safety (Vincent et al., 1998) and Donabedian's conceptual model for assessing quality (Donabedian, 1988). The full version of the questionnaire included 60 items, of which only 30 were standard (Sexton et al., 2006a). The SAQ version used in this study consists of 33 items of which three non-standard items. The non-standard items are used because they are relevant for the purpose of our future research program (items 14, 26, 33). Within these 33 items, 5 items are subdivided into 2 sub-items each, measuring the perception of “hospital management” and “unit management”. The instrument elicits caregiver attitudes through six factor-analytically derived scales: teamwork climate, job satisfaction, perception of management, safety climate, working conditions, and stress recognition. Each of the items is answered using a five-point Likert scale (disagree strongly, disagree slightly, neutral, agree slightly, agree strongly and not applicable as score 6). However, to our knowledge, the content validity of the original SAQ has never been tested in a cross-cultural language version. Several studies measured the internal consistency of the instrument and the scales in different settings, with Cronbach's alpha values for the different scales ranging from 0.68 to 0.89 (Deilkas and Hofoss, 2008, Kaya et al., 2010, Modak et al., 2007, Nordén-Hägg et al., 2010). Construct validity of the SAQ was also assessed, using confirmatory factor analyses and demonstrating a satisfactory model fit across languages and settings (Deilkas and Hofoss, 2008, Modak et al., 2007, Nordén-Hägg et al., 2010, Sexton et al., 2006a).

The purpose of this study was to explore the psychometric properties of the SAQ Dutch language version by evaluating its internal consistency, content and face validity.

Section snippets

Phase 1: translation of the SAQ

The SAQ (Sexton et al., 2006a) was translated (forward translation) into Dutch by three of the authors (E. Dev., J.C. and K.M.) with excellent knowledge of English (Cha et al., 2007). Instead of using backward translation, a panel of 15 Dutch-speaking experts were selected by four of us (E. Dev., J.C., K.V. and K.M) of which 7 researchers with five having nursing, one having medical and one having psychological backgrounds and eight clinicians with four nurses having clinical and management

Phases 1 and 2: translation and content validity of the SAQ

Twelve of the 15 experts evaluated the translation of the SAQ; one clinician did not respond and 2 experts of which one researcher and one clinician found themselves not capable enough to evaluate the scale. As a consequence, 12 experts agreed on most of the proposed translated items. However, a subset of items (n = 12) had to be reformulated to provide a translation that was valid in the Belgian hospital context.

Twelve of the 15 experts rated the content validity of the Dutch version of the SAQ.

Discussion

The purpose of this study was to assess the validity and reliability of the Dutch version of the Safety Attitudes Questionnaire (SAQ). In line with evidence from other cross-cultural studies (Deilkas and Hofoss, 2008, Kaya et al., 2010, Modak et al., 2007, Nordén-Hägg et al., 2010, Sexton et al., 2006a), our findings mostly support the validity of the SAQ. However, our study went further to use experts to evaluate content validity while replicating the standard psychometric tests. Experts

Conclusion

We conclude that in this study the Dutch version of the Safety Attitudes Questionnaire showed acceptable to good psychometric properties. In line with the previous evidence, this instrument seems to be an acceptable to good tool to evaluate the safety climate. However, some items of the SAQ translated in Dutch should be should be evaluated in future studies and may have to be modified or removed from the instrument. Our data suggest the need for more extensive validation in different settings.

Authors’ contributions

Els Devriendt, Koen Van den Heede, Joke Coussement, Eddy Dejaeger, Kurt Surmont, Dirk Heylen, René Schwendimann, and Koen Milisen substantially contributed towards conception and design; Els Devriendt, Kurt Surmont, and Koen Milisen were responsible for acquisition of data; Els Devriendt, Koen Van den Heede, René Schwendimann, Bryan Sexton, Nathalie Wellens, Steven Boonen, and Koen Milisen did study analysis and interpretation of data; Els Devriendt and Koen Milisen drafted the article; Els

Ethical approval

The study was approved by the Medical Ethics Committee of the Leuven University Hospitals.

Conflict of interest

None declared.

Funding

None declared.

Acknowledgements

We thank the experts and all nurses, physicians and physiotherapist of the nine wards of UZ Leuven for completing questionnaires. Dr. Boonen is senior clinical investigator of the Fund for Scientific Research-Flanders, Belgium (F.W.O.-Vlaanderen) and holder of the Leuven University Chair in Gerontology and Geriatrics.

References (36)

  • P.J. Pronovost et al.

    Improving patient safety in intensive care units in Michigan

    Journal of Critical Care

    (2008)
  • Agency for Healthcare Research and Quality, 2011. Hospital survey on patient safety culture. Rockville, MD....
  • B.T. Carney et al.

    The effect of facility complexity on perceptions of safety climate in the operating room: size matters

    American Journal of Medical Quality

    (2010)
  • E.S. Cha et al.

    Translation of scales in cross-cultural research: issues and techniques

    Journal of Advanced Nursing

    (2007)
  • D.V. Cicchetti et al.

    Developing criteria for establishing interrater reliability of specific items: application to assessment of adaptive behaviour

    American Journal of Mental Deficiency

    (1981)
  • J.B. Colla et al.

    Measuring patient safety climate: a review of surveys

    Quality and Safety in Healthcare

    (2005)
  • E.T. Deilkas et al.

    Psychometric properties of the Norwegian version of the Safety Attitudes Questionnaire (SAQ), Generic version (Short form 2006)

    BMC Health Service Research

    (2008)
  • E.N. De Vries et al.

    The incidence and nature of in-hospital adverse events: a systematic review

    Quality and Safety in Healthcare

    (2008)
  • X. Feng et al.

    Patient safety culture in nursing: a dimensional concept analysis

    Journal of Advanced Nursing

    (2008)
  • A. Field

    Reliability analysis

    Discovering Statistics Using SPSS

    (2009)
  • A. Donabedian

    The quality of care. How can it be assessed?

    Journal of the American Medical Association

    (1988)
  • Fleis

    Statistical Methods for Rates and Proportions

    (1981)
  • D.J. France et al.

    Measuring and comparing safety climate in intensive care units

    Medical Care

    (2010)
  • Health and Safety Commission

    Organizing for Safety: Third Report of the Human Factors Study Group of ACSNI

    (1993)
  • J. Hellings et al.

    Kwaliteit inzake medicatiebeleid in ziekenhuizen: een onderzoek naar de kritische succesfactoren in de samenwerking tussen artsen, apothekers, verpleegkundigen en managers

    (2009)
  • L.M. Holden et al.

    Patient safety climate in primary care: age matters

    Journal of Patient Safety

    (2009)
  • D.T. Huang et al.

    Intensive care unit safety culture and outcomes: a US Multicenter Study

    International Journal of Quality in Health Care

    (2010)
  • D.W. Hudson et al.

    A safety culture primer for the critical care clinician: the role of culture in patient safety and quality improvement

    Contemporary Critical Care

    (2009)
  • Cited by (38)

    • Psychometrical evaluation of an instrument to measure the nurses’ knowledge about early mobilization among post laparotomy patients

      2021, Enfermeria Clinica
      Citation Excerpt :

      A previous study conducted also showed Cronbach's alpha value of 0.90.20 Other studies by developing and validating psychometric scales to prevent early diarrhea in children obtained Cronbach's alpha (0.84), which shows that the scale is reliable.21 Coefficient results for items between 0.63 and 0.95 with p < 0.05 indicating high internal reliability with Cronbach's alpha values varying between 0.63 and 0.85 and 0.89 so they can be used as instruments.22

    • Measuring the safety climate in an Australian emergency department

      2021, International Emergency Nursing
      Citation Excerpt :

      The Safety Climate Survey (SCS) [26] is a 43-item questionnaire adapted from the Safety Attitudes Questionnaire (SAQ) [27] for use in the Australian healthcare setting. The SCS was selected based upon its appropriateness to the study setting and the strength of previous psychometric testing for the SAQ (scale reliability α = 0.9 [28], and content validity index of 0.83 [29]). Permission to adapt and use the SCS was granted by the Victorian Managed Insurance Authority in October 2017.

    • The perception of the patient safety climate by professionals of the emergency department

      2017, International Emergency Nursing
      Citation Excerpt :

      Researchers have demonstrated the importance of assessing safety culture with instruments capable of measuring the safety climate at various levels, in order to obtain useful information about the areas that need to be improved [11,12]. While the terms “safety culture” and “safety climate” are often used interchangeably, in the present study, the term “safety climate” was adopted, defined as a temporal state measure of safety culture in institutions [13–15]. The most common way to measure safety climate is the numerical method, which is performed by using scales and questionnaires that assess attitudes, teamwork climate, job satisfaction, working conditions and individual stress factors [15,16].

    • Safety climate in the operating room: Translation, validation and application of the Safety Attitudes Questionnaire

      2016, Revista Portuguesa de Saude Publica
      Citation Excerpt :

      The values obtained in the study of validity of the instrument both in each factor and as a whole are of the same magnitude of the figures presented by the authors of the questionnaire.19 Translations and adaptations of “Safety Attitudes Questionnaire” for other languages also revealed a high content validity.23,24 The SAQ has also been extensively used to relate climate safety with the results for the patient,24 however this study did not address this issue.

    View all citing articles on Scopus
    1

    Alternative address: Katholieke Universiteit Leuven, Centre for Health Services and Nursing Research, Kapucijnenvoer 35, 5th floor, 3000 Leuven, Belgium. Tel.: +32 16 336906; fax: +32 16 336970. E-mail address: [email protected].

    View full text