Content validity and internal consistency of the Dutch translation of the Safety Attitudes Questionnaire: An observational study
Introduction
Patient safety is fundamental to healthcare quality and is on the research agenda of numerous policy institutions in many countries. Attention has recently focused on the patient safety culture of an organisation and its impact on patient outcomes (Colla et al., 2005, Huang et al., 2010, Hudson et al., 2009). According to the World Health Organisation, every point in a process of care-giving contains a certain risk of inherent unsafety (WHO, 2011). Studies demonstrate that about one out of ten hospitalized patients in Western countries are affected by adverse events. This is an injury secondary to medical care and not a result of a patient's underlying medical condition (Institute of medicine, 2000). Approximately one-half of the adverse events was considered preventable (De Vries et al., 2008).
A strong safety culture is essential for in-hospital patient safety and reduces the risk of adverse events (Kline et al., 2008). Patient safety culture is defined as a “subset of organisational culture, which relates specifically to the values and beliefs concerning patient safety within healthcare organisations” (Feng et al., 2008). In essence, culture can be expressed as “the way we do things around here on our work unit” (Pronovost and Sexton, 2005). A more readily measurable aspect of safety culture at the local workplace is the ‘climate of patient safety’, and was described as “the product of individual and group values, attitudes, perceptions, competencies, and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation's health and safety management” (Health and Safety Commission, 1993). However, climate and culture are used interchangeably in the literature.
Patient safety culture/climate measures are widely used as diagnostic tools to identify areas for improvement of care processes. It can also be an onset for patient safety programs which can raise awareness about the role of an appropriate culture in promoting a safer patient environment (Nieva and Sorra, 2003, Pronovost and Sexton, 2005).
A variety of instruments exists to measure an organisation's patient safety climate such as the Safety Attitudes Questionnaire (Sexton et al., 2006a), the Hospital Survey on Patient Safety Culture (Agency for Healthcare Research and Quality, 2011) or the Safety Organising Scale (Vogus and Sutcliffe, 2007). These instruments measure domains of safety culture such as management/supervisors, safety systems, risk perception, job demands, reporting/speaking up, safety attitudes/behaviours, communication/feedback, teamwork, personal resources and organisational factors (Hellings and Vleugels, 2009). The strengths of these tools vary and the SAQ is recommended by several authors (Sexton et al., 2006a, Modak et al., 2007, Deilkas and Hofoss, 2008, Kaya et al., 2010, Nordén-Hägg et al., 2010, Poley et al., 2011) because of the following reasons. The validity and reliability of the SAQ has been documented in many countries such as the United States (Modak et al., 2007, Sexton et al., 2006a), Norway (Deilkas and Hofoss, 2008), Turkey (Kaya et al., 2010), Sweden (Nordén-Hägg et al., 2010) and the Netherlands (Poley et al., 2011). Furthermore, the instrument has also been adapted for the use in different settings and specializations such as intensive care units (ICUs) (France et al., 2010, Pronovost et al., 2008, Poley et al., 2011, Sexton et al., 2011), operating rooms (Carney et al., 2010), general inpatient settings such as medical and surgical wards (Deilkas and Hofoss, 2008), ambulatory clinics (Holden et al., 2009, Modak et al., 2007), pharmacies (Nordén-Hägg et al., 2010), and labour and delivery units (Sexton et al., 2006b). Also the SAQ has been extensively used to explore the relationship between safety climate scores and patient outcomes, e.g. medication errors, adverse events,… (Colla et al., 2005, Huang et al., 2010, Hudson et al., 2009). Finally, all versions of the SAQ have the same contents for the various items, with minor modifications to reflect the clinical area. Because of the above-mentioned reasons we have chosen to translate this instrument and validate the Belgian version of the Dutch SAQ.
The Safety Attitudes Questionnaire (SAQ) developed by Sexton et al. (2006a) is based on two conceptual models: Vincent's framework for analysing risk and safety (Vincent et al., 1998) and Donabedian's conceptual model for assessing quality (Donabedian, 1988). The full version of the questionnaire included 60 items, of which only 30 were standard (Sexton et al., 2006a). The SAQ version used in this study consists of 33 items of which three non-standard items. The non-standard items are used because they are relevant for the purpose of our future research program (items 14, 26, 33). Within these 33 items, 5 items are subdivided into 2 sub-items each, measuring the perception of “hospital management” and “unit management”. The instrument elicits caregiver attitudes through six factor-analytically derived scales: teamwork climate, job satisfaction, perception of management, safety climate, working conditions, and stress recognition. Each of the items is answered using a five-point Likert scale (disagree strongly, disagree slightly, neutral, agree slightly, agree strongly and not applicable as score 6). However, to our knowledge, the content validity of the original SAQ has never been tested in a cross-cultural language version. Several studies measured the internal consistency of the instrument and the scales in different settings, with Cronbach's alpha values for the different scales ranging from 0.68 to 0.89 (Deilkas and Hofoss, 2008, Kaya et al., 2010, Modak et al., 2007, Nordén-Hägg et al., 2010). Construct validity of the SAQ was also assessed, using confirmatory factor analyses and demonstrating a satisfactory model fit across languages and settings (Deilkas and Hofoss, 2008, Modak et al., 2007, Nordén-Hägg et al., 2010, Sexton et al., 2006a).
The purpose of this study was to explore the psychometric properties of the SAQ Dutch language version by evaluating its internal consistency, content and face validity.
Section snippets
Phase 1: translation of the SAQ
The SAQ (Sexton et al., 2006a) was translated (forward translation) into Dutch by three of the authors (E. Dev., J.C. and K.M.) with excellent knowledge of English (Cha et al., 2007). Instead of using backward translation, a panel of 15 Dutch-speaking experts were selected by four of us (E. Dev., J.C., K.V. and K.M) of which 7 researchers with five having nursing, one having medical and one having psychological backgrounds and eight clinicians with four nurses having clinical and management
Phases 1 and 2: translation and content validity of the SAQ
Twelve of the 15 experts evaluated the translation of the SAQ; one clinician did not respond and 2 experts of which one researcher and one clinician found themselves not capable enough to evaluate the scale. As a consequence, 12 experts agreed on most of the proposed translated items. However, a subset of items (n = 12) had to be reformulated to provide a translation that was valid in the Belgian hospital context.
Twelve of the 15 experts rated the content validity of the Dutch version of the SAQ.
Discussion
The purpose of this study was to assess the validity and reliability of the Dutch version of the Safety Attitudes Questionnaire (SAQ). In line with evidence from other cross-cultural studies (Deilkas and Hofoss, 2008, Kaya et al., 2010, Modak et al., 2007, Nordén-Hägg et al., 2010, Sexton et al., 2006a), our findings mostly support the validity of the SAQ. However, our study went further to use experts to evaluate content validity while replicating the standard psychometric tests. Experts
Conclusion
We conclude that in this study the Dutch version of the Safety Attitudes Questionnaire showed acceptable to good psychometric properties. In line with the previous evidence, this instrument seems to be an acceptable to good tool to evaluate the safety climate. However, some items of the SAQ translated in Dutch should be should be evaluated in future studies and may have to be modified or removed from the instrument. Our data suggest the need for more extensive validation in different settings.
Authors’ contributions
Els Devriendt, Koen Van den Heede, Joke Coussement, Eddy Dejaeger, Kurt Surmont, Dirk Heylen, René Schwendimann, and Koen Milisen substantially contributed towards conception and design; Els Devriendt, Kurt Surmont, and Koen Milisen were responsible for acquisition of data; Els Devriendt, Koen Van den Heede, René Schwendimann, Bryan Sexton, Nathalie Wellens, Steven Boonen, and Koen Milisen did study analysis and interpretation of data; Els Devriendt and Koen Milisen drafted the article; Els
Ethical approval
The study was approved by the Medical Ethics Committee of the Leuven University Hospitals.
Conflict of interest
None declared.
Funding
None declared.
Acknowledgements
We thank the experts and all nurses, physicians and physiotherapist of the nine wards of UZ Leuven for completing questionnaires. Dr. Boonen is senior clinical investigator of the Fund for Scientific Research-Flanders, Belgium (F.W.O.-Vlaanderen) and holder of the Leuven University Chair in Gerontology and Geriatrics.
References (36)
- et al.
Improving patient safety in intensive care units in Michigan
Journal of Critical Care
(2008) - Agency for Healthcare Research and Quality, 2011. Hospital survey on patient safety culture. Rockville, MD....
- et al.
The effect of facility complexity on perceptions of safety climate in the operating room: size matters
American Journal of Medical Quality
(2010) - et al.
Translation of scales in cross-cultural research: issues and techniques
Journal of Advanced Nursing
(2007) - et al.
Developing criteria for establishing interrater reliability of specific items: application to assessment of adaptive behaviour
American Journal of Mental Deficiency
(1981) - et al.
Measuring patient safety climate: a review of surveys
Quality and Safety in Healthcare
(2005) - et al.
Psychometric properties of the Norwegian version of the Safety Attitudes Questionnaire (SAQ), Generic version (Short form 2006)
BMC Health Service Research
(2008) - et al.
The incidence and nature of in-hospital adverse events: a systematic review
Quality and Safety in Healthcare
(2008) - et al.
Patient safety culture in nursing: a dimensional concept analysis
Journal of Advanced Nursing
(2008) Reliability analysis
Discovering Statistics Using SPSS
(2009)
The quality of care. How can it be assessed?
Journal of the American Medical Association
Statistical Methods for Rates and Proportions
Measuring and comparing safety climate in intensive care units
Medical Care
Organizing for Safety: Third Report of the Human Factors Study Group of ACSNI
Kwaliteit inzake medicatiebeleid in ziekenhuizen: een onderzoek naar de kritische succesfactoren in de samenwerking tussen artsen, apothekers, verpleegkundigen en managers
Patient safety climate in primary care: age matters
Journal of Patient Safety
Intensive care unit safety culture and outcomes: a US Multicenter Study
International Journal of Quality in Health Care
A safety culture primer for the critical care clinician: the role of culture in patient safety and quality improvement
Contemporary Critical Care
Cited by (38)
Psychometrical evaluation of an instrument to measure the nurses’ knowledge about early mobilization among post laparotomy patients
2021, Enfermeria ClinicaCitation Excerpt :A previous study conducted also showed Cronbach's alpha value of 0.90.20 Other studies by developing and validating psychometric scales to prevent early diarrhea in children obtained Cronbach's alpha (0.84), which shows that the scale is reliable.21 Coefficient results for items between 0.63 and 0.95 with p < 0.05 indicating high internal reliability with Cronbach's alpha values varying between 0.63 and 0.85 and 0.89 so they can be used as instruments.22
Measuring the safety climate in an Australian emergency department
2021, International Emergency NursingCitation Excerpt :The Safety Climate Survey (SCS) [26] is a 43-item questionnaire adapted from the Safety Attitudes Questionnaire (SAQ) [27] for use in the Australian healthcare setting. The SCS was selected based upon its appropriateness to the study setting and the strength of previous psychometric testing for the SAQ (scale reliability α = 0.9 [28], and content validity index of 0.83 [29]). Permission to adapt and use the SCS was granted by the Victorian Managed Insurance Authority in October 2017.
The perception of the patient safety climate by professionals of the emergency department
2017, International Emergency NursingCitation Excerpt :Researchers have demonstrated the importance of assessing safety culture with instruments capable of measuring the safety climate at various levels, in order to obtain useful information about the areas that need to be improved [11,12]. While the terms “safety culture” and “safety climate” are often used interchangeably, in the present study, the term “safety climate” was adopted, defined as a temporal state measure of safety culture in institutions [13–15]. The most common way to measure safety climate is the numerical method, which is performed by using scales and questionnaires that assess attitudes, teamwork climate, job satisfaction, working conditions and individual stress factors [15,16].
Safety climate in the operating room: Translation, validation and application of the Safety Attitudes Questionnaire
2016, Revista Portuguesa de Saude PublicaCitation Excerpt :The values obtained in the study of validity of the instrument both in each factor and as a whole are of the same magnitude of the figures presented by the authors of the questionnaire.19 Translations and adaptations of “Safety Attitudes Questionnaire” for other languages also revealed a high content validity.23,24 The SAQ has also been extensively used to relate climate safety with the results for the patient,24 however this study did not address this issue.
- 1
Alternative address: Katholieke Universiteit Leuven, Centre for Health Services and Nursing Research, Kapucijnenvoer 35, 5th floor, 3000 Leuven, Belgium. Tel.: +32 16 336906; fax: +32 16 336970. E-mail address: [email protected].