Impact of system-level activities and reporting design on the number of incident reports for patient safety
- 1Department of Healthcare Economics and Quality Management, School of Public Health, Kyoto University Graduate School of Medicine, Kyoto, Japan
- 2Institute for Health Economics and Policy, Tokyo, Japan
- Correspondence to Professor Yuichi Imanaka, Department of Healthcare Economics and Quality Management, School of Public Health, Kyoto University Graduate School of Medicine, Yoshida Konoe-cho, Sakyo-ku, Kyoto 606-8501, Japan;
- Accepted 26 November 2008
Background Incident reporting is a promising tool to enhance patient safety, but few empirical studies have been conducted to identify factors that increase the number of incident reports.
Objective To evaluate how the number of incident reports are related to system-level activities and reporting design.
Methods A questionnaire survey was administered to all 1039 teaching hospitals in Japan. Items on the survey included number of reported incidents; reporting design of incidents; and status for system-level activities, including assignment of safety managers, conferences, ward rounds by peers, and staff education. Staff education encompasses many aspects of patient safety and is not limited to incident reporting. Poisson regression models were used to determine whether these activities and design of reporting method increase incident reports filed by physicians and nurses.
Results Educational activities were significantly associated with reporting by physicians (53% increase, p<0.001) but had no significant effect on nurse-generated reports. More reports were submitted by physicians and nurses in hospitals where time involved with filing a report was short (p<0.05). The impact of online reporting was limited to a 26% increase in physicians' reports (p<0.05).
Conclusion In accordance with the suggestions by previous studies that examined staff perceptions and attitudes, this study empirically demonstrated that to decrease burden to reporting and to implement staff educations may improve incident reporting.
- Patient safety
- incident reporting
- reporting systems
- national survey
- teaching hospital
- health professions education
Despite emphasis by the Institute of Medicine on the importance of building a safer healthcare system,1 study of effectiveness of safety programmes on patient safety has not yet accelerated. Because of lack of evaluation tools and difficulty in measuring rare outcomes over short periods for small samples of patients with progressive diseases,2–4 trials based on adverse events as the outcome can be extraordinarily difficult.5 Instead, using systematic quality improvement, which has demonstrated success in non-medical industries, is a promising approach.5 6
As quality and safety enhancement involves the ability to learn from errors, which have emphasised consequences in high-risk industries,7–9 incident reporting is a leading initiative proposed to improve patient safety.1 10–12 Having a systematic collection of reports enables organisational learning by identification of sources of failure and thereby allows implementation of corrective actions. Under-reporting, however, is inherently involved in incident reporting systems. Because under-reporting of incidents has been estimated to range from 50% to 96%,13–19 the frequency of reports likely does not represent true incidence of errors. Therefore, in using numbers of incident reports as the indicator of capacity of organisational learning, we must adjust barriers to reporting.
In the present study, we identified factors that increase the number of incident reports based on aspects of system-level activities for patient safety and design of incident reporting method.
Data and sampling
We conducted our cross-sectional survey between December 2006 and May 2007, using all 1039 teaching hospitals in Japan. Ownership structures of teaching hospitals vary widely in Japan, and our survey included university, national, municipal, public and corporate models of ownership. Number of acute care beds varied from 42 to 1505 per hospital.
This study was approved by the institutional review board at the Graduate School of Medicine of Kyoto University.
Development of questionnaire
The questionnaire was designed to collect information about numbers of incident reports filed by physicians and nurses, system-level activities for patient safety and design of incident reporting method. Questionnaire content was developed based on a review of past literature on patient safety20–24 and clinical experience of a multidisciplinary panel of healthcare professionals and patient safety experts. The questionnaire included factors addressed in previous studies, most of which examined staff perceptions of barriers to reporting incidents: a busy and fatigued workforce,8 9 25–41 fear of reporting,8 9 28–38 42 and lack of knowledge about reporting.8 9 23 25–27 30–32 35–43
Because each question in the questionnaire was designed to investigate the method of current incident reporting, and the system-level activity status of each hospital is stringently recorded, the reliability of the responses may be thought of as high. In addition, the questionnaire was validated through interviews with several managers of patient safety and discussions with panels of experts. Because our validation process involved literature reviews and expert consensus, we believe that the survey questions have at least face validity and are reliable markers of patient safety systems.
Dependent variables: numbers of incident reports
We measured numbers of incidents by type of professionals (physicians and nurses) reporting incidents during the 6 months from April to September 2006. As the hospital accreditation authority in Japan has a voluntary incident reporting system using a three-level classification scheme based on severity of injury, most hospitals refer to any of the following criteria as an incident: (1) an event occurred, but was caught before reaching the patient; (2) an event occurred and reached the patient, but patient was unharmed; and (3) an event occurred and the patient was affected, but the treatment attributable to the event was minimal. Incidents that involve a degree of harm to patients that requires more than the minimal amount of treatment are categorised as accident and therefore were not included in our definition of incidents.
Independent variables: predictors of numbers of incident reports
We controlled for certain hospital characteristics that were suspected to be confounding factors in counting the number of incident reports submitted. These characteristics included ownership and the number of inpatient-days.
In addition, the length of time that the incident reporting was in place at each hospital was included because of the hypothesis that the longer an incident reporting system has been in place, the higher the chance that the individual staff members understand the importance and methods to report incidents. We used this factor as a binary variable: below and above the median of elapsed years from implementation.
Design of incident reporting method
To control some suggested barriers to reporting incidents, patient safety managers were asked to evaluate the following four qualitative measures of design of incident reporting method: (1) whether the incidents were reported electronically or via paper; (2) average length of time staff spent to fill out an incident report; (3) whether a policy of non-punitive reporting was guaranteed by written documentation and/or orally; (4) numbers of recommendations per bed that the hospital implemented to improve systems, processes or products resulting from incidents reported between April and September 2006.
The first question refers to the potential barrier of a cumbersome method of reporting, which may negatively affect staff perception of reporting.25–30 The emphasis on the barrier of perceived staff busyness and fatigue is inherent in the second question. If the potential reporter is too busy and too tired to report incidents, decreasing extra work involved in reporting is important.8 9 30–41 We required respondents to answer which of the following lengths of time was closest to the average for reporting: ≤15 min, ≤30 min, ≤45 min, ≤60 min or >60 min. Responses then were collapsed into two categories: ≤ 30 and >30 min.
Confidentiality or immunity from punishment may be essential for potential reporters to overcome the barrier of fear.8 9 28–38 42 Therefore, we gave two options for the third aspect of reporting design—a policy of non-punitive reporting ensured by written documentation and/or oral description, or no policy. The fourth aspect—recommendations derived from reported incidents—was based on past findings that giving feedback on results of incident reporting is useful to enhance reporting.25–33 41
Amount of system-level activities for patient safety
Because lack of knowledge about the reporting method is one of the most important barriers to reporting incidents,23 25–27 30–32 35–43 we evaluated the amount of staff education for physicians and nurses. Staff education encompasses many aspects of patient safety and is not limited to incident reporting. In addition, we assessed activities to advance the “plan-do-check-act” cycle and, thereby, to improve patient safety systems, including assignment of safety managers, conferences and ward rounds. The person-time spent on these practices was calculated for a specified 6-month window (table 1). To calculate the person-time for each patient safety activity, we surveyed number of staff, amount of time required per activity session and frequency of activity sessions conducted between April and September 2006. Then, we converted the time spent by personnel on patient safety programmes into 2007 US dollars, using the employee's hourly wage44–46 and the Purchasing Power Parities.47 Finally, by use of the number of beds and the distribution of amount of each system-level activities among respondent hospitals, we collapsed the cost per 100 beds into two categories: over and under the median.
We excluded hospitals lacking data regarding incident reports, reporting design or institutional characteristics. In the remaining hospitals, the top 1% of hospitals in terms of numbers of incident reports was further excluded from analysis because we found that these hospitals showed inordinately high incident report numbers and would therefore act as outliers that substantially affect the estimates of regression analysis. Because numbers of incident reports conform to a Poisson distribution, Poisson regression with overdispersion was used to perform multivariable analysis. Standard errors were made heteroskedastically consistent via the Huber–White covariance matrix. Stata V.9.2 was used for all analyses.
A total of 418 hospitals participated in the study (response rate, 40.2%). Hospitals that did not meet inclusion criteria were dropped from the statistical analysis, resulting in a final sample of 232 hospitals.
The mean (SE) of incident reports per 10 000 inpatient-days by physicians and nurses was 2.62 (0.18) and 91.3 (4.42), respectively. Table 2 compares reporting design of incidents. Although 63.8% of hospitals surveyed required an average of ≤30 min to report incidents, approximately 80 hospitals (36.2%) took an average of >30 min to fill out a report. Significantly more hospitals used paper-based reporting than online reporting (65.5% vs 34.5%; p<0.001).
The median (interquartile range) dollars spent on system-level activities per 100 beds during a 6-month period for assigning patient safety managers, conferences, ward rounds by peers, education for physicians and education for nurses was $9410 ($5729–$13 575), $1326 ($873–$1899), $204 ($79–$482), $992 ($50–$3440), and $488 ($63–$1128), respectively (table 3).
Results of regression analyses presented in table 4 demonstrate that incident reports filed by physicians could be increased by online reporting (26%, p<0.05) and shorter time required to file a report (27%, p<0.05). Moreover, hospitals that implemented more education for physicians significantly increased reporting by 53% (p<0.001). In hospitals with dedicated full-time staff for the purpose of patient safety, the number of incident reports by physicians significantly increased by 35% (p<0.05). However, immunity policy and rate of recommendations derived from reported incidents did not significantly influence the number of physician-generated incident reports.
However, results of predictor factors in numbers of nurse-reported incidents, compared to physician-reported incidents, showed different relationships. Nurse-reported incidents were encouraged only by decreased time for reporting (22% increase, p<0.05). Education for nurses was not a significant factor in reporting.
There was no significant relationship between the elapsed years of incident reporting system and the number of incidents reported by physicians and nurses.
To our knowledge, this is the first empirical study that explores determinants associated with incident reporting and identifies the impact of system-level activities on numbers of incident reports that could increase capacity of organisational learning. Our results provide new evidence supporting most of the previous studies that examined staff perceptions regarding incident reporting.
Our rationale for the present study was an extension of other works hypothesising that system-level activities enable reporting of incidents by establishing a solid safety culture among employees. In the first outcome studies in patient safety, researchers focused on factors that contribute to improvement of safety culture. Ginsburg et al21 and Thomas et al22 found that an improved safety culture was associated with implementation of staff education and executive ward rounds, respectively. Next, Naveh et al48 empirically demonstrated that enhanced safety culture was associated with increased reporting of incidents. Later, a randomised controlled study by Figueiras et al23 showed that physician-generated reporting of adverse drug reactions was increased by implementing staff education. Recent studies revealed that implementation of a multifaceted intervention package comprising staff education and changes in reporting designs could improve incident reporting.25 26 30 Because previous studies did not assess the effectiveness of each patient safety programme on incident reporting, we investigated these issues in the current study.
In contrast to the physician-generated reports, there was no significant association between education for nurses and the number of nurse-initiated reports. A possible reason for this difference is a decreased marginal effect of education for nurses. Considering that the average number of nurse-generated reports was more than 30 times higher than that of physicians and that educational time for nurses was more than seven times greater than that for physicians, nurses' knowledge about incident reporting appears to be sufficient. According to previous studies,31 36 37 41 staff perspectives regarding reporting show that lack of knowledge was not a major deterrent for reporting by nurses, although it may be a major barrier to reporting by physicians. In addition, other studies demonstrated that implementing physician education resulted in significant improvement in reporting by physicians.23 38 Therefore, our results were partially consistent with previous qualitative findings. Furthermore, we shed light on the impact of assignment of safety manager on the number of incident reports filed by physicians for the first time. Although our survey focused on system-level activities for patient safety that were conducted organisationally, other daily detailed activities not specifically included in the survey were instead covered under the duties of full-time dedicated staff. For example, each dedicated staff member performs the activities involved in analysing reported incidents and giving feedback on results of incident reporting, and promotes awareness of patient safety throughout hospital via such routine activities. These activities might affect physicians' attitude to incident reporting.
In addition to a lack of knowledge as discussed above, our result that busyness and fatigue are barriers of incident reporting was consistent with past literature that examined the reasons of under-reporting of incidents.31–41 By decreasing the time to fill out incident reports, the number of incidents by physicians and nurses could be significantly increased by 27% and 22%, respectively. Meanwhile, the influence of reporting method (online vs paper-based) was different depending on type of professions, and this was again in concordance with previous findings. For example, the finding that physician-generated online reports significantly increased by 26% (p<0.05) is in accordance with previous studies that examined numbers of incidents reported via online systems.28 29 Regarding reporting by nurses, in contrast, our results are also similar to previous studies.29 30 Perceptions of usefulness versus the cumbersome nature of online reporting might depend on accessibility and a user-friendly interface. Because reducing the time required to fill out a report would obviously make reporting less burdensome, this reporting design could generate more reports by physicians and nurses.
Although fear of reporting has been previously found to be another barrier,31–38 42 our studies did not observe this result. Willingness to report incidents could depend on the legislative system, such as presence of laws protecting patient safety whistle-blowers from retaliation. Japanese healthcare providers are susceptible to criminal prosecution for professional negligence.49 Therefore, the impact of an immunity policy in hospitals might decrease the barrier of fear in reporting incidents.
Previous studies that evaluated staff perception suggested that giving feedback to staff was useful to enhance reporting.27 31–33 A possible reason why our study did not confirm a significant relationship between these factors is that because our study was an observational study and therefore unable to standardise the definition of recommendation derived from reported incidents, there would be discrepancy of content of feedback among participant hospitals. Further study is needed to examine the true impact of feedback on incident reporting.
Previous studies have paid little attention to the impact of elapsed years since implementation of a reporting system, whereas it is likely that the longer an incident reporting system has been in existence in a particular hospital might correlate with a better maturation of the reporting system and therefore present an increased number of incidents. Although our model was unable to demonstrate this relationship, this result should be viewed with caution. Because the variable in our model used the elapsed years of the first adoption of incident reporting system, it did not reflect that of the current reporting system.
Our study has several limitations. First, questions on the amount of system-level activities were answered by patient safety managers. Therefore, even if activities to improve patient safety systems were conducted within other departments, all activities implemented in a hospital might not be reflected in our survey and thus might diminish the measurable effect of the activities. Second, many hospitals did not respond to our questionnaire, thus raising the existence of selection bias. Those that participated in our survey may systematically establish patient safety systems as compared to hospitals that did not respond our questionnaire because it is likely that hospitals that recorded daily activity status may tackle the issue of patient safety in an organised way. Therefore, our results might reflect the status of teaching hospitals with relatively high motivation to enhance patient safety. Last, our findings could be favourable within a stage in which the incident reporting system has not been fully matured. When the understanding of the incident reporting system increases among professions, and each staff member will report all incidences that he or she encounter, the next stage will be that the number of true incidents will gradually diminish. Although the influence of secular trends on our results is unclear, our findings may offer an effective way to attain such a desirable subsequent stage.
In conclusion, our results demonstrate empirically that the number of incident reports reflect the degree of staff education and have implications for initiatives to design better reporting methods. Further research is needed to develop successful educational content and to modify incident reporting formats. A challenge is to balance competing goals of ease of the reporting process and the need for more detailed information enabling prevention of recurrence of similar incidents.
The authors thank all the hospitals that participated in this research.
Funding The work described in this article was funded in part by the Health Sciences Research Grants for the Research on Policy Planning and Evaluation from the Ministry of Health, Labor and Welfare of Japan and the Grant-in-aid for Scientific Research A from the Ministry of Education, Culture, Sports, Science and Technology of Japan.
Competing interests None.
Ethics approval This study was conducted with the approval of the institutional review board at the Graduate School of Medicine of Kyoto University.
Provenance and peer review Commissioned; externally peer reviewed.