Incident reporting is a tool that enables healthcare workers to disclose adverse events, defined as unintended injury caused by healthcare management rather than the patient’s disease,^1,2 and near misses. Information from reports enables contributing factors to be identified and corrections made to prevent recurrence of similar incidents.² With recognition that hospitals should establish effective voluntary incident management systems,^3–5 it is important that these meet the needs of those expected to use them.

There is evidence from studies comparing incident reporting with other techniques to detect adverse events^6,7 and from clinicians themselves^8,9 that many incidents are not reported. A number of studies have attempted to improve incident reporting^10–15; however, a literature review has failed to identify any studies aimed at improving incident reporting using the rigorous study design of a matched controlled study.

This intervention aimed to improve reporting and stimulate a greater variety of reported incidents by doctors and nurses, and assess the effectiveness of components of the intervention.

METHODS

Study population

This study was undertaken in South Australia, Australia, which contains a population of approximately 1.5 million people and 78 acute public hospitals (12 major city, 44 regional and 22 remote hospitals).¹⁶ Nursing and medical staff working in 10 control and intervention units across two regional and four major city tertiary hospitals participated in the study. Table 1 gives the details of the characteristics of these hospitals and units.

Study design

The two regional hospitals were purposively selected on the basis of their similar sizes¹⁶ and geographical isolation from each other. One of the regional hospitals employed doctors to work in its emergency department (ED), whereas in the other hospital general practitioners (GPs) worked on a rotational basis in the ED. In both hospitals, inpatient care was provided by GPs. To reduce contamination of the control and intervention units caused by doctors working across multiple inpatient areas, units were stratified into inpatient areas (including the intensive care unit (ICU), medical and surgical units) and EDs, and were block randomised.

In major city hospitals, intervention and control units were matched by type of unit (medical units, surgical units, ICUs and EDs) and services provided (neurology, cardiology and gastrointestinal surgery). Difficulty in recruiting necessitated non-random allocation of control and intervention units, in order to ensure a spread of units across hospitals.

This study compared incident-reporting rates and types of reports generated between (a) baseline and study period and between (b) control and intervention units. Intervention rollout was staggered, from June to August 2003, with units implementing the intervention over a 40-week period.

Components of the intervention

We conducted focus groups using a topic guide with 14 doctors and 19 nurses⁸ and surveyed doctors and nurses across four major cities and two regional hospitals (n = 773, response rate = 72.8%) using a questionnaire⁹ to identify barriers to reporting and current reporting practices. Validation of the questionnaire and the methods for conduct and analysis of the focus groups has been reported previously.^8,9 This information guided the intervention design.

Results showed that many doctors did not know what or how to report, and that senior doctors particularly feared legal repercussions. Reporting was largely seen as the nurses’ responsibility. Fear of disciplinary action, time constraints and lack of feedback were other identified barriers. In addressing these barriers, the following intervention components were developed:

Education

A manual was developed to improve knowledge of reportable incidents and to address fear of disciplinary action. Manuals were distributed to all GPs, medical heads of units, nurse unit managers and patient safety managers, and placed in tearooms and medical resident’s rooms. A detailed verbal explanation of all aspects of the study was provided to all medical heads of units, nurse unit managers and patient safety managers, and to most GPs (project officers were unable to schedule a meeting with 20% of GPs). Posters explaining the types of incidents to be reported were strategically placed in handover rooms, medical staff rooms and nurses’ stations. Incident forms were prominently displayed in the clinical areas.

Education sessions were scheduled in existing departmental meetings to explain the purpose of the study, changes in reporting processes, the ability to report anonymously, the importance of reporting near misses and qualified privilege afforded under the Commonwealth legislation to identify information disclosed through the reporting system.

Reduce fear of reporting

To address the concern that some nurse unit managers used reports against staff, reporting processes were redesigned (fig 1). Reports were initially sent to the patient safety manager. Identified reports had the severity of the incident validated and action taken by the nurse unit managers for nurse-generated reports, and by the medical head of unit for doctor-generated reports. Anonymous reports were validated and managed only by the patient safety manager in each institution.

Each patient safety manager, medical head of unit and nurse unit manager from intervention units was encouraged to attend a two-day root cause analysis (RCA) workshop, where they were taught how to investigate incidents effectively.¹⁷

Reduce reporting burden

A one-page report form replaced the existing three-page form. A freecall telephone service (1800NOTIFY) was introduced, enabling staff to report an incident 24 h a day, 7 days a week, to a registered nurse, who entered incident details directly into the statewide Advanced Incident Management System (AIMs) database. Online reporting was piloted in the ICU of major cities, in addition to the call centre and paper-based reporting.

Improve feedback

Four newsletters were distributed to doctors and nurses. These newsletters contained statistics, summaries of quality-improvement investigations, and de-identified RCA findings and recommendations from incidents reported in intervention units during the study period. Individual feedback was provided for incidents classified by reporters as resulting in death, permanent loss/lessening of function, additional surgery or increased length of stay. For less serious events, aggregate data were presented at departmental meetings scheduled at least every 3 months.

Statistical analysis

We assumed a type 1 error of 0.05, 90% power and a two-sided test. Then, if the control group reporting rate was 100/10 000 occupied bed days (OBDs), a sample size of 10 375 OBDs in each group would be sufficient to demonstrate a 50% increase in reporting rate in the intervention group compared with the control group as being statistically significant. This requirement is reduced by the matched design, but increased because of the within-hospital clustering. Further, a number of subgroup analyses were considered necessary. Hence, a study period was chosen to provide approximately 35 000 OBDs in each study arm at baseline and during the study period.

The relative change in reporting rates from baseline to the end of the study between the control and intervention units was assessed using Fisher’s exact test. Reporting rates by type of unit (medical, surgical and ED) and location of hospital (major city and regional) were compared using negative binomial regression analysis, adjusting for clustering by hospitals. Poisson regression analysis using robust variance was used to assess changes in the ICU and designation of reporter by profession, since negative binomial regressions would not converge. Comparisons between intervention and control units adjusted for differences at baseline were undertaken by using an interaction term as the measure of effect.

A principal incident type was allocated for each incident by coders, who were blinded to the location of the incident. Changes in types of incidents reported were analysed using log binomial regression analysis. Principal incident types were classified a priori into those likely and unlikely to implicate the nurse unit manager. The conventional level of p⩽0.05 was taken to represent statistical significance. Analysis was done using the STATA statistical software package V.7.0. Approval was obtained from each of the participating hospital’s ethics committee.

RESULTS

Reporting rate

Despite matching by type of unit and hospital location, there was considerable heterogeneity between control and intervention units at baseline. Owing to the large size of the major city intervention ICU, there were twice as many ICU beds in intervention units (42 beds) as those in control units (21 beds), a significant difference (χ²₍₁₎ = 11.19, p = 0.001; table 1). At baseline, there were significantly more reports made in inpatient intervention units and in ED control units, and there were more anonymous reports generated in control units (table 2).

Compared with control units, the intervention resulted in an absolute increase of 60.3 reports/10 000 OBDs (95% CI 23.8 to 96.8 reports/10 000 OBDs) in inpatient areas (p<0.001), 39.5 reports/10 000 ED attendances (95% CI 17.0 to 62.0 reports/10 000 ED attendances; p<0.001) and 20.2 anonymous reports/10 000 ED attendances and OBDs combined (95% CI 12.6 to 27.8 reports/10 000 ED attendances and OBDs combined; p<0.001). Within inpatient areas, the most significant improvement occurred in medical units, with an additional 84.5 reports/10 000 OBDs (95% CI 24.9 to 144.1 reports/10 000 OBDs). The intervention was not able to demonstrate improved reporting in the ICUs. There was heterogeneity between individual units, with rates in medical and surgical intervention units (n = 6) ranging from 113 to 431 reports/10 000 OBDs.

Reporter designation

Most of the reports were generated by nurses (84%), followed by allied health professionals (11%) and doctors (5%). The intervention did not significantly improve reporting by doctors in inpatient areas, but did improve reporting in EDs. Conversely, reporting by nurses improved in the inpatient areas, but not in the EDs (table 3). With regard to anonymous reports, in 63 reports, the reporters identified only their profession (8 doctors and 55 nurses), 22 reports contained only the location of the incident and in 101 reports neither was disclosed. In hospitals with only one intervention unit (hospitals 1, 4 and 5), no paper reports were submitted anonymously.

Reporting format

The majority (79%) of the 1275 reports lodged during the study period in intervention units were reported using the single-page form, with 21% submitted through the call centre (table 4). The average time taken to report an incident through the call centre was 9 min 20 s; 57% of reports were logged out of office hours and 24% made on the weekend. Staff in major city hospitals used the call centre significantly more often than their regional counterparts (32.1% vs 4.1%, p<0.001). Regional units were the first to implement call centre reporting. Poor call centre use by regional hospitals is likely due to protracted report-taking by a few inexperienced call centre operators in the first few weeks of commencing the project. More documentation-related reports (13.1% vs 4.9% of all reports, p<0.001) and less human performance-related reports (2.3% vs 7.2% of all reports, p<0.001) were generated using the one-page form compared with the call centre method.

Despite providing training and giving access to staff, only six reports were lodged on-line in the ICU. There was no significant difference by profession as to the reporting format used.

Change in types of incidents reported

Table 5 shows the comparison of principal incident types between intervention and control units at the end of the study period, having adjusted for baseline reporting practices. Despite falls-related reports doubling in intervention units, as a percentage of all incidents reported, there was a significant relative decline (36.1–23.8% of all reports, p<0.001). Intervention units reported increased numbers of documentation, clinical management, aggression and environmental incidents during the study period, compared with their baseline-reporting practices and reporting in the control unit. More incidents were submitted anonymously for incidents in which line managers were often implicated (clinical management, organisational management, and behaviour and human performance incidents) compared with identified reports (17.9% vs 11.7%, p = 0.009).

Of the 57 doctor-initiated reports made during the study period, most related to clinical management issues (n = 18) and medication errors (n = 13). Because the majority of all reports were submitted by nurses, the overall changes in incident types, shown in table 5, largely reflect those of nurses.

Addressing barriers to reporting

Education

One education session was provided in each unit for doctors prior to commencing of the project. A number of sessions over a 2-week period were provided to nursing staff. Allied health workers attended departmental meetings, interacted with staff and were exposed to promotional posters and incident forms displayed in intervention units, which may have improved their reporting from a very low baseline (table 3).

All nurse unit managers and patient safety managers, and three of the six major medical heads of units in the city attended the RCA workshop. Despite offering financial incentive (lost wages and all costs associated with attending), no GPs attended. Reporting by doctors in units where the medical heads of units attended the workshop was higher than in areas where this did not occur (35 vs 14 reports).

Feedback

There were 185 incidents in which severity was not classified by the reporter. Individual feedback was provided to reporters of the 41 (3.7%) incidents that resulted in death or permanent loss of function, and to reporters of the 90 (8.0%) incidents that caused permanent lessening of function, additional surgery or increased length of stay. With the exception of the regional ICU, where feedback was given on a 3-monthly basis, aggregate feedback to nurses of less serious incidents by project officers was provided on a monthly basis in all intervention units in sessions at shift change. Feedback to doctors at departmental meetings was more difficult. In the four regional units, despite a number of attempts, only two feedback sessions were provided, because of a lack of participation by GPs. Units receiving ⩾3 feedback sessions in the 40-week study period included the major city ED and two major city medical units. These three units generated more doctor-generated reports than the combined total of the seven other intervention units.

DISCUSSION

The intervention resulted in more anonymous reports and more reports in EDs, medical and surgical units, but not in ICUs. There were a greater number of doctor-initiated reports in the ED and more nurse-generated reports in areas other than the ED.

The lack of ability to improve reporting in ICUs may be due to the fact that, unlike other successful ICU reporting studies,^10,12 project officers were unable to provide intense education, facilitation and feedback to the 200 nurses and 40 doctors. The lack of doctor-initiated reports meant that feedback was limited to discussion of incidents generated in other areas, with potentially limited perceived relevance. Attributes of those units and groups demonstrating significant improvement in reporting are outlined in box 1.

Although other studies to improve incident reporting have had greater success than our study in inpatient areas^7,11 and the ED,^18,19 without a control unit for comparison, it is difficult to attribute the improvement to the intervention. With regard to reporting within professional groups, our doctor-initiated reporting rate was lower than studies targeting only doctors using intense facilitation,^10,15,20 and yet comparable with¹⁴ or higher than studies not restricted to doctors.^21,22 As demonstrated in other studies targeting ⩾1 professional groups, the majority of reports in our study were generated by nurses.^11,14

In our study, one-fifth of all reports were generated using the call centre, which is higher than the 9% identified in a US study in primary practice.¹³ As with other studies in institutions offering a variety of reporting options, when given a choice, staff prefer to use paper-based reporting to online reporting.

Possible reasons for the doubling of reports in surgical and medical control units during the study period include contamination by intervention unit staff and allied health professionals working in control units as part of normal hospital activity, and the impact of other quality activities in the hospital at the time of the intervention, including state-wide RCA training.

The ability of the study to facilitate reporting of a more diverse range of incidents is important. In a climate in which staff are constantly facing time pressures, it is important that resources are focused on reporting incidents for which there is limited knowledge about contributing factors and in which important lessons can be learned.

Even though our intervention actively provided feedback and highlighted the ability to report anonymously, it lacked elements identified in other successful reporting systems.²³ Reports were not sent to an independent organisation.²⁴ Most reports were managed by nurse unit managers and medical heads of units who were not provided with additional time, financial assistance, or a risk register to assist in prioritising and taking action on the many more reports being generated. Unlike aviation,²⁵ there was no consistent systematic evaluation of reports at a macro-level by an independent expert trained at identifying underlying causes of incidents in the system.

There were a number of limitations to our study. Contamination of results was suggested by the finding that 15 call centre incidents and five one-page reports were generated by staff in control units. Such contamination diminished the measurable effect of the intervention. Lack of randomisation in major city hospitals might have introduced selection bias. Baseline reporting in the intervention ICU was higher than in its control unit, whereas baseline reporting in the control ED was higher than in its intervention unit. Our analysis focused on the relative change between baseline and study period, to take into account the observed baseline differences. To reduce any selection bias associated with non-randomised sampling of major city units, wherever possible, we matched unit types within the hospital (eg, matching medical subspecialty with medical subspecialty) and specialties between hospitals (eg, neurology unit in hospital 4 with neurology unit in hospital 2). However, units with higher reporting rates at baseline may be more amenable to strategies to increase reporting, or units with lower reporting rates at baseline may have greater potential for improvement.

Our finding that it is possible to improve reporting rates and diversify the types of incidents reported is encouraging. If voluntary reporting is not successful in identifying adverse events, particularly by doctors, then there is the real risk that governing bodies will not continue to support it, or that a more regulatory approach will be taken instead.