You are here

Article Text

Article menu

Download PDFPDF

Error management
Reporting of Clinical Adverse Events Scale: a measure of doctor and nurse attitudes to adverse event reporting
  1. B Wilson1,
  2. H L Bekker2,
  3. F Fylan3
  1. 1
    Department of Health Sciences, University of York, Heslington, York, UK
  2. 2
    Institute of Health Sciences and Public Health Research, School of Medicine, University of Leeds, Leeds, UK
  3. 3
    Department of Psychology, School of Health and Human Sciences, Leeds Metropolitan University, Leeds, UK
  1. Dr B Wilson, Research Associate, School of Nursing, Midwifery and Social Work, The University of Manchester, Oxford Road, Manchester M13 9PL, UK; barbara.wilson{at}christie-tr.nwest.nhs.uk

Abstract

Objective: To develop a validated measure of professionals’ attitudes towards clinical adverse event reporting (CAER).

Design: Cross-sectional survey with follow-up.

Participants: 201 doctors and nurse/nurse-midwives undergoing postqualification training in Leeds, York and Hull Universities in 2003.

Materials: A questionnaire which comprised 73 items extracted from interviews with professionals; a second, statistically reduced version of this questionnaire.

Results: The analysis supported a 25-item questionnaire comprising five factors: blame as a consequence of reporting (six items); criteria for reporting (six items); colleagues’ expectations (six items); perceived benefits of reporting events (five items); and clarity of reporting procedures (two items). The resulting questionnaire, the Reporting of Clinical Adverse Effects Scale (RoCAES), had satisfactory internal consistency (Cronbach’s α = 0.83) and external reliability (Spearman’s correlation = 0.65). The construct validity hypothesis—doctors have less positive attitudes towards CAER than nurses—was supported (t = 5.495; p<0.0001).

Conclusion: Initial development of an evidence-based, psychometrically rigorous measure of attitudes towards CAER has been reported. Following additional testing, RoCAES may be used to systematically elicit professionals’ views about, and inform interventions to improve, reporting behaviour.

https://doi.org/10.1136/qshc.2006.021691

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Clinical adverse events reporting (CAER) is a key aspect of the National Health Service (NHS) clinical governance12 and is essential to improve the safety and quality of services by enabling staff to learn from previous errors.1–3 Being able to identify incidents causing most detriment to patients—adverse events—means that scare resources aimed at preventing reoccurrence through root cause analysis can be focused on those incidents impacting most heavily on patient outcomes. In the UK, the National Reporting and Learning System (NRLA) is being developed to enable healthcare staff to record events and identify error patterns needing investigation and correction. However, previous research suggests that many events go unreported,4–6 jeopardising the validity of such schemes.7–9 In addition, substantial variation in CAER behaviour between professional groups is apparent: doctors are less likely to report an adverse event than nurses and midwives.10–14 Proposals to increase CAER have been guided by the apparent success of reporting programmes in other occupations, including aviation and the nuclear power industry,3 and revolve around development of a “reporting culture” within the NHS.

    Qualitative studies have identified a number of barriers hindering the development of this reporting culture. Principal among these is the idea of a “blame culture,” with staff concerned that they will be individually held responsible for errors and disciplined. Other barriers include: fear of reactions of co-workers; the characteristics of specific events; lack of feedback.1012–19 The culture of medicine itself has also been highlighted as a significant barrier, with its shared beliefs about collegiality, self-regulation and the inevitability of errors providing ubiquitous discouragement to staff reporting.15

    The majority of interventions attempting to increase CAER address: administrative and/or employment regulations; the information required or format of report forms; and reducing disciplinary action against health professionals who report.20 Yet few interventions have been developed that have explicitly and rigorously targeted the attitudes and/or culture of health professionals influencing CAER behaviour. It could be argued that the development of an effective attitude-based intervention, in which appropriate attitudes were targeted, is likely to be the most effective means of changing attitudes. Such an intervention would require a valid means of measuring any changes in attitudes that had been achieved. However, to our knowledge, there is no psychometrically rigorous measure of health professionals’ attitudes to CAER with which to inform and/or evaluate the effectiveness of any intervention aimed at changing CAER within the NHS. The aim of this study is to address this shortfall through assessing the validity and reliability of a questionnaire developed to measure health professionals’ views towards CAER: item development; factor analysis using principal axis factoring; assessment of the questionnaire’s internal, external reliability and construct validity are discussed.

    METHODS

    Design

    Cross-sectional survey with follow-up.

    Sample

    The sample comprised nurses/nurse-midwives and doctors undergoing postqualification training at Leeds, Hull and York Universities in August/September 2003.

    Materials

    Study information was developed in accordance with research ethics guidelines. The questionnaire assessed the following areas.

    • demographic characteristics, that is age, gender, length of time qualified, professional role.

    • behaviour: two items assessing individual’s experience of witnessing and reporting adverse events, and their intention to report future adverse events.

    • attitudes: 73 statements about aspects of reporting adverse events which were identified from the analysis of an interview study with 11 doctors and nine nurses across Radiology, Medical Oncology, Ophthalmology and General Medicine specialties in a northern city hospital.18

    Each statement had a corresponding 4-point Likert rating scale (strongly agree, agree, disagree and strongly disagree), with higher scores indicating a more negative attitude towards reporting. Items included both positive and negative statements, and were reverse-scored where appropriate.

    Procedure

    Course leaders were contacted about the study, and a date and time was identified to distribute the initial long version of the questionnaire. Participation was voluntary. Participation was anonymous unless participants agreed to complete the second, shorter questionnaire at a second time-point, in which case they provided their name and address. After factor analysis, the second, statistically reduced questionnaire was posted to those participants agreeing to complete it, 6 weeks after completion of the first questionnaire.

    Analysis

    Data were analysed using SPSS (version 10). Factor analysis using principal axis factoring (PAF) with orthogonal (varimax) and oblique (direct obliminal) rotation techniques was employed to assess internal consistency—how items measuring similar concepts group together. Factor scores were calculated from statements within each dimension by summing all items within a dimension, having reverse-scored negative-phrased statements. Construct validity—how scale scores relate to other findings—was assessed by testing the hypothesis that professional groups have different CAER beliefs: prior research suggests that nurses and doctors have different CAER behaviours and are likely, therefore, to hold different views. A t test was undertaken to test this hypothesis. To assess external reliability, Spearman’s correlation coefficient was applied to the five factor scores calculated from those respondents who completed both the original and statistically reduced questionnaires.

    RESULTS

    A total of 201 health professionals completed the questionnaire: 122 (61%) were nurses/nurse-midwives, 75 (37%) were doctors, and four (2%) did not state their profession. The majority (65%) of respondents were female, and the mean age was 37 (SD 7.3) years. The mean time since primary qualification was 14 (7.5) years. The 25-item, statistically reduced questionnaire was returned by 74 (37%) participants: 44 (60%) were nurses and nurse-midwives, and 29 (40%) were doctors; one (1%) did not state their profession.

    Internal consistency reliability

    Orthogonal principal axis factoring of the 201 responses was undertaken. All 73 items were assessed for item redundancy through the factor analysis: communalities less than 0.2 were removed to aid interpretability of the final factor solution; multicollinearity and singularity were assessed by examining the determinant of the R-matrix, which indicated that these were not present in the data (determinant = 9.570E−18). Applying a cut-off factor loading score of 0.364 and deleting double-load items generated 25 items loading on five coherent factors. These accounted for 41% of the variance. The five factors represented separate dimensions of attitudes towards CAER: blame as a consequence of reporting (six questions), criteria for reporting events (six questions), colleagues’ expectations (six questions), perceived benefits of reporting events (five questions) and clarity of reporting procedures (two questions). The factor loadings for each item are shown in table 1.

    View this table:
    Table 1 Factor loadings for each of the 25 attitude items in Reporting of Clinical Adverse Effects Scale (RoCAES)

    Cronbach’s α coefficient was satisfactory (ie, ⩾0.65) for each of the five coherent dimensions, and the overall internal consistency for the five-factor questionnaire was 0.83 (table 2).

    View this table:
    Table 2 Assessment of internal consistency reliability using coefficient alpha

    External reliability

    Just over half of respondents (52%) agreed to complete a second questionnaire. The 25-item, statistically reduced questionnaire Reporting of Clinical Adverse Effects Scale (RoCAES) was sent to 102 health professionals 6 weeks after they completed the first questionnaire, and 74 (73%) were returned. Comparing the responses from the 25-item, statistically reduced questionnaire with the same items of the original 73-item questionnaire using Spearman’s correlation coefficient showed a moderate reliability or consistency for the overall questionnaire (0.65) and individual dimensions (table 3)

    View this table:
    Table 3 Six-week retest reliability coefficients

    Construct validity

    No statistically significant difference was found between nurses/nurse-midwives and doctors’ experience of witnessing an adverse events (88% vs 85%; p = 0.994). Most respondents (85%) had witnessed an adverse event, and of these, the majority (89%) had reported it. Nurses/nurse-midwives were more likely to report an adverse event than doctors (95% vs 80%; p = 0.001). Nurses and nurse midwives had a significantly more positive attitude towards reporting than doctors on each of the five dimensions on the first and longer questionnaire (table 4). On both versions of the questionnaire, nurses and nurse midwives had a more positive attitude towards reporting than doctors (p<0.05).

    View this table:
    Table 4 Differences in attitudes towards reporting in doctors and nurses/nurse midwives

    DISCUSSION

    Development and preliminary evidence on the validity and reliability of a measure of health professionals’ attitudes towards reporting adverse events has been described. The questionnaire—RoCAES—comprises 25 items that assess attitudes towards reporting. These items load onto five factors: blame as a consequence of reporting; perceived criteria for which incidents should be reported; colleagues’ expectations; perceived benefits of reporting adverse events; and perceived clarity of reporting procedures. In addition, two items address previous reporting behaviour, and intentions to report future events.

    Initial support for the validity of the questionnaire is apparent: the dimensions identified in the analyses reflect the range of beliefs found in qualitative studies of CAER1214–16 indicating the measure successfully taps complex attitudes. For example: “Perceived blame’ is made up of items focusing on the potential (negative) consequences of CAER for an individual, corresponding to previously identified beliefs around blaming;12 “Perceived criteria for which incidents should be reported” includes items tapping the differential criteria used by staff to decide what to report, corresponding with findings showing the frequency and consequences of events for patients’ health affect reporting;141617 “Perceptions of colleagues’ expectations” includes items dealing with how colleagues view reporting, tapping professional cultural beliefs.15 In keeping with prior research, RoCAES identified doctors as having less positive attitudes towards CAER than nurses/nurse-midwives.1011 External reliability scores for each dimension and the overall questionnaire were moderately correlated.

    Clearly, these findings describe the preliminary stages of RoCAES development. Further psychometric testing is required, recommendations for which include: application to a more diverse staff population to access the views of those with more varied CAER behaviour; test–retest of the 25 item, statistically reduced questionnaire as a complete instrument, rather than through comparison of the same items within the long and shorter versions of the scale. In addition, researchers may be able to refine the psychometric properties of the RoCAES by operationalising additional concepts, including that of “shame,” requiring a further assessment of individual item correlation to determine item redundancy. This study described the preliminary development of a measure to assess staff attitudes to CAER. RoCAES has the potential to become an essential instrument in the training of NHS staff around issues of CAER and medical professionalism.21 The results from RoCAES applications can be used to help focus interventions designed to change NHS CAER culture and behaviour, and to evaluate the success of interventions. As such, RoCAES has implications for current NHS safety and care objectives: its ability to systematically measure staff views around reporting potentially aids the development of a reporting culture within the service; a prerequisite to the ability of the NRLA to reliably reflect current risks to patient safety, and for the NHS as a whole to successfully prevent the reoccurrence of events through organisational learning.

    Acknowledgments

    The authors thank Dr J Miles, Lecturer in Biostatistics, University of York, for statistical advice, and Dr M Campbell, Lecturer in Statistics, University of Manchester, for statistical advice.

    Appendix A Reporting of Clinical Adverse Events Scale questionnaire

    The statements listed below have been used by other professionals to describe their views on clinical adverse event reporting. Please read each statement. Circle the number to the right of each statement that best describes your opinion of the statement. A “clinical adverse event” is any aspect of the healthcare process that may impact negatively on patients, which is not part of the progress of their illness or a result of their own actions.

    View this table:

    • Have you ever witnessed, or been involved in, an adverse event? (tick one box): yes □ no □

    • If yes, have you ever reported an adverse event? (tick one box): yes □ no □

    • How likely are you to report an adverse event in the future? very unlikely □ unlikely □ neither likely/nor unlikely □ likely □ very likely □

    Thank you for filling out this questionnaire.

    REFERENCES

    1. ↵
      Department of Health. The new NHS: modern, dependable. London: TSO, 1997.
    2. ↵
      Department of Health. A first class service: quality in the new NHS. London: TSO, 1998.
    3. ↵
      Department of Health. An organisation with a memory. London: TSO, 2000.
    4. ↵
      1. Stanhope N,
      2. Crowley-Murphy M,
      3. Vincent C,
      4. et al
      . An evaluation of adverse incident reporting. J Eval Clin Pract 1999;5:5–12.
      OpenUrlCrossRefPubMedWeb of Science
      1. Barach P,
      2. Small S
      . Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ 2000;320:759–63.
      OpenUrlFREE Full Text
    6. ↵
      1. Schectman JM,
      2. Plews-Ogan ML
      . Physician perception of hospital safety and barriers to incident reporting. Joint Comm J Qual Patient Safety 2006;32:337–43.
      OpenUrl
    7. ↵
      1. In: Kohn L T,
      2. Corrigan J M,
      3. Donaldson M S
      , eds. To err is human. Building a safer health system. Washington: Committee on Quality of Health Care in America, Institute of Medicine, National Academy Press, 1999.
      1. Rosenthal J,
      2. Riley T
      . Patient safety and medical errors: a road map for state action. Portland. National Academy of State Health Policy. 2001. http://www.nashp.org (accessed 27 Jan 2008).
    9. ↵
      1. Sari AB,
      2. Sheldon TA,
      3. Cracknell A,
      4. et al
      . Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. BMJ 2007;334:79–81.
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Vincent C,
      2. Stanhope N,
      3. Crowley-Murphy M
      . Reasons for not reporting adverse incidents: an empirical study. J Eval Clin Pract 1999;5:13–21.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Tuttle D,
      2. Holloway R,
      3. Baird T,
      4. et al
      . Electronic reporting to improve patient safety. Qual Saf Health Care 2004;13:281–6.
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Firth-Cozens J,
      2. Redfern N,
      3. Moss F
      . Confronting errors in patient care. 2002. http://www.publichealth.bham.ac.uk/psrp/pdf/Focusgroupreport_firthcozens.pdf (accessed 27 Jan 2008).
      1. Cullen D,
      2. Bates D,
      3. Small S,
      4. et al
      . The incident reporting system does not detect adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv 1995;21:541–8.
      OpenUrlPubMed
    14. ↵
      1. Evans SM,
      2. Berry JG,
      3. Smith BJ,
      4. et al
      . Attitudes and barriers to incident reporting: a collaborative hospital study. Qual Saf Health Care 2006;15:39–43.
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Waring JJ
      . Beyond blame: cultural barriers to medical incident reporting. SocSciMed 2005;60:1927–35.
      OpenUrl
    16. ↵
      1. Orbe MP,
      2. King G III
      . Negotiating the tension between policy and reality: exploring nurses’ communication about organizational wrongdoing. Health Comm 2000;12:41–61.
      OpenUrlCrossRef
    17. ↵
      1. Lawton R,
      2. Parker D
      . Barriers to incident reporting in a healthcare system. Qual Saf Health Care 2002;11:15–18.
      OpenUrlAbstract/FREE Full Text
    18. ↵
      1. Cullen DJ,
      2. Bates DW,
      3. Leape LL,
      4. et al
      . Prevention of adverse drug events: a decade of progress in patient safety. Clin Anesthesia 2000;12:600–14.
      OpenUrlCrossRef
    19. ↵
      1. Westwood M,
      2. Rodgers M,
      3. Sowden A
      . Patient safety: a mapping of the research literature. York: NHS CRD, University of York, 2002.
    20. ↵
      1. Wilson B
      . Clinical adverse event reporting: development and validation of an attitude scale [dissertation]. York, University of York, 2006.
    21. ↵
      General Medical Council, Education Committee. Tomorrows’ doctors: recommendations on undergraduate medical education: issued by the Education Committee of the General Medical Council in pursuance of section 5 of the Medical Act 1983. London: General Medical Council, 1993.

    Footnotes

    • Funding: Department of Health Sciences, University of York PhD Scholarship. The author’s work is independent of the funding source.

    • Competing interests: None.

    • Ethics approval: Approval was obtained from module tutors to contact students in their classes. Local ethics committee approval was not required, as the study was undertaken in August/September 2003 with a student sample.

    • Contributors: BW (Guarantor); HLB; FF—each fulfils the criteria of authorship as laid out in the uniform requirements for manuscripts submitted to medical journals. No other person or persons fulfil this requirement.