Objective: This study determined the incidence, type, nature, preventability and impact of adverse events (AEs) among hospitalised patients and potentially preventable deaths in Dutch hospitals.
Methods: Using a three-stage retrospective record review process, trained nurses and doctors reviewed 7926 admissions: 3983 admissions of deceased hospital patients and 3943 admissions of discharged patients in 2004, in a random sample of 21 hospitals in the Netherlands (4 university, 6 tertiary teaching and 11 general hospitals). A large sample of deceased patients was included to determine the occurrence of potentially preventable deaths in hospitals more precisely.
Results: One or more AEs were found in 5.7% (95% CI 5.1% to 6.4%) of all admissions and a preventable AE in 2.3% (95% CI 1.9% to 2.7%). Of all AEs, 12.8% resulted in permanent disability or contributed to death. The proportion of AEs and their impact increased with age. More than 50% of the AEs were related to surgical procedures. Among deceased hospital patients, 10.7% (95% CI 9.8% to 11.7%) had experienced an AE. Preventable AEs that contributed to death occurred in 4.1% (95% CI 3.5% to 4.8%) of all hospital deaths. Extrapolating to a national level, between 1482 and 2032 potentially preventable deaths occurred in Dutch hospitals in 2004.
Conclusions: The incidence of AEs, preventable AEs and potentially preventable deaths in the Netherlands is substantial and needs to be reduced. Patient safety efforts should focus on surgical procedures and older patients.
Statistics from Altmetric.com
Previous retrospective record review studies in several countries have shown that 2.9% to 16.6% of patients in acute care hospitals experience one or more adverse events (AEs) and in 4.5–20.8% of the AEs, the patient dies.1–13 Approximately 50% of the AEs are judged to be preventable. An AE is defined as an unintended injury that results in temporary or permanent disability, death or prolonged hospital stay, and is caused by healthcare management rather than by the patient’s underlying disease process.25813 Variation in the incidence of AEs will be partly determined by policies and characteristics at hospital level (within country variation, between hospitals), partly by policies at national level (between country variation) and partly by differences in methodology of the studies.1415
The occurrence of AEs in Dutch hospitals has not been systematically studied. Insight in preventable AEs can increase the sense of urgency and offer a starting point for specific interventions to improve patient safety, whereas insight in unpreventable AEs may help prioritise research areas. Therefore, we conducted a retrospective patient record review study to determine the incidence, type, nature, impact and preventability of AEs among hospitalised patients in the Netherlands. This study is the largest population-based study carried out in Europe on the occurrence of AEs and reflects the increasing attention paid to patient safety across Europe. The protocol and instruments were based on the Canadian Adverse Events Study.2
Box 1 Definitions and outcome measures16
Adverse event determination
AE determination was based on the presence of three criteria:
an unintended (physical and/or mental) injury, which
resulted in temporary or permanent disability, death or prolongation of hospital stay, and was
caused by healthcare management rather than the patient’s disease.
To determine whether the injury was caused by healthcare management or the disease process (criterion 3) a six-point scale was used:
(Virtually) no evidence for management causation
Slight to modest evidence of management causation
Management causation not likely (less than 50/50, but “close call”)
Management causation more likely (more than 50/50, but “close call”)
Moderate to strong evidence of management causation
(Virtually) certain evidence of management causation
Causation scores of 4–6 were classified as AEs.
Timing of AEs
The index hospital admission was the admission sampled. AEs were included if they occurred during the index admission and were detected during or within 12 months after the index admission. AEs were also included if they were related to hospital admissions in the same hospital in the 12 months preceding the index admission, but were not detected until the index admission.
The degree of preventability of AEs was measured on a six-point scale, grouped into three categories:
1. (Virtually) no evidence for preventability
2. Slight to modest evidence of preventability
3. Preventability not quite likely (less than 50/50, but “close call”)
4. Preventability more than likely (more than 50/50, but “close call”)
5. Strong evidence of preventability
6. (Virtually) certain evidence of preventability
AEs with a preventability score of 4–6 were defined as preventable AEs.
Potentially preventable hospital deaths
Potentially preventable hospital deaths were defined as highly preventable AEs which contributed to death during the hospital admission. The adjective “potentially” is used because of the multifactorial nature of hospital deaths and the retrospective assessment of causality.
The doctor reviewers estimated the life expectancy of the deceased patients for the situation that the hospital admission would have evolved without an AE, taking into account the health status of the patient as described in the patient records and using their professional judgement.
Study design and setting
The design and methods have been described in detail elsewhere.16 We performed a retrospective patient record review study in a random stratified sample of 21 of the 101 Dutch hospitals: 4 university, 6 tertiary teaching and 11 general hospitals. To measure the difference in incidence between hospital types, the sample of hospitals was stratified for hospital type. Proper representation of urban and rural settings in the sample was verified. Eligible hospitals had at least 200 beds, an emergency department and an intensive care unit. A large subsample of deceased hospital patients was included to determine more precisely than in previous studies the occurrence of potentially preventable deaths in hospitals.
The power calculation of this study was based on the results of the Canadian Adverse Events Study.2 Assuming an incidence of AEs of 8%, we required a sample of 4200 hospital admissions of discharged patients and a sample of 4200 admissions of deceased patients (β = 0.20, α = 0.05) to estimate a 95% confidence interval of 0.5% on both sides. To measure the difference in incidence between hospital types, a selection of 800 hospital admissions per hospital type was necessary to detect a difference from 2% to 3% by an incidence between 3% and 7%.
From each hospital, we randomly selected 200 admissions (>24 h stay) of discharged patients and 200 (or fewer if the total of patients who died in 2004 was lower) admissions of deceased hospital patients in 2004, excluding admissions to psychiatry and obstetrics and of children <1 year.16
Review of patient records
The nursing, medical and, if available, outpatient record of the sampled admissions were reviewed by 66 trained nurses and 55 trained doctors in a three-stage review process between August 2005 and October 2006. In the first stage, a nurse screened the records by using 18 screening criteria indicating potential AEs (see online appendix A). In the second stage, two doctors independently reviewed the records with one or more positive screening criteria. Based on a standardised procedure they determined presence, nature, impact, clinical process and degree of preventability of the AEs. Also the life expectancy in case the AE had not occurred was estimated for deceased patients with AEs (box 1). If there was disagreement about the presence and/or preventability of an AE between the two doctors’ reviews, they undertook a consensus procedure (stage 3). If they could not reach a consensus, a third trained doctor reviewer gave the final judgement.16
To assess the reliability of the first stage of the record review process, a random sample of 415 records was independently reviewed by a second nurse. To assess the reliability of the final judgement (including any consensus and third review if applicable) of the doctors’ review, a random sample of 119 records was reviewed by a second pair of doctors.16
We calculated the national weighted incidence of AEs and preventable AEs in Dutch hospitals with 95% confidence intervals, corrected for the over-representation of patients admitted to a university hospital and for the over-representation of patients who died in hospital, using the “complex sample” option in SPSS V14.0. The sample weight was the inverse of the probability of being included in the sample owing to the sample design. It was calculated as N/n, where N = the number of elements in the population and n = the number of elements in the sample (see online appendix B). After weighting for the sampling frame, the total study sample (discharged and deceased patients) and the subsample of deceased patients were representative for the Dutch population of hospitalised patients and for the Dutch population of deceased hospital patients, respectively (see online appendix C). The characteristics of the AEs, such as disability and classification, and patient and admission characteristics were also assessed using weights to adjust for the sampling frame. The analysis for the subsample of deceased patients was only weighted for the over-representation of patients admitted to a university hospital.
The inter-rater agreement of the review process between nurses for finding screening criteria and for the determination of AEs and their preventability between two pairs of doctors was expressed as a κ statistic with 95% confidence intervals and as a percentage of records for which there was agreement.
Incidence of AEs among hospitalised patients
Of the 8415 sampled records, 8032 were eligible for a first-stage review (screening success rate was 95%) (fig 1). In 54% of these records one or more screening criteria were identified (appendix A). In the second stage, the reviewing doctors identified 744 AEs in 663 hospital admissions. In 70 hospital admissions (10.6% of 663), two or more AEs were found. In all, 504 AEs were found in 447 deceased and 240 AEs in 216 discharged patients. After weighing for the sampling frame, the national incidence of AEs among hospitalised patients was 5.7% (95% CI 5.1% to 6.4%) and the incidence of preventable AEs was 2.3% (95% CI 1.9% to 2.7%) (table 1).
Incidence rates of AEs were significantly higher in university hospitals than in general hospitals (p<0.001). Although not statistically significant, the incidence of preventable AEs was lower in university hospitals compared with the other types of hospital.
Of 744 AEs, 39.6% were considered preventable. More than half of all AEs (56.8%) resulted in no or minimal physical impairment or disability. However, 5.0% resulted in permanent disability and 7.8% contributed to death. The incidence of AEs was higher in surgical departments than in non-surgical departments (table 2).
A quarter of all AEs occurred in the previous 12 months and were detected in the index admission in the same hospital. The majority of AEs (63%) occurred and were detected during the index admission, whereas 12% occurred during the index admission and were detected within 12 months after the index admission.
More than half (54%) of all AEs were related to surgical procedures. Almost all AEs related to the diagnostic process were highly preventable and 23% contributed to death (table 3). The proportion of AEs, preventable AEs and degree of disability increased with age (table 4).
Incidence of AEs among deceased patients
The incidence of AEs and preventable AEs in the subsample of deceased patients was 10.7% (95% CI 9.8% to 11.7%) and 5.2% (95% CI 4.5% to 5.9%), respectively (table 1). The incidence of preventable AEs contributing to death among deceased patients was 4.1% (95% CI 3.5% to 4.8%). In 2004, 42 329 patients died in Dutch hospitals, amounting to 1735 (95% CI 1482 to 2032) potentially preventable hospital deaths.
Compared with the total hospital population, AEs among deceased patients were more often preventable (47.7% vs 39.6%) and were more often related to the diagnostic process (14.8% vs 6.3%). Almost half (49.1%) of the deceased patients with a potentially preventable AE that contributed to death had an estimated potential life expectancy of more than 1 year had the potentially preventable AE not have occurred. In 11% of the cases the life expectancy could not be determined (table 5).
Agreement between reviewers
The reliability of the assessment of screening criteria by nurses was good (κ 0.62; 95% CI 0.54 to 0.69; 82% agreement). The reliability of determination of AEs was only fair (κ 0.25; 95% CI 0.05 to 0.45; 76% agreement), and it was moderate for determination of preventability of AEs (κ 0.40; 95% CI 0.07 to 0.73; 70% agreement).
The present study found that in 5.7% of all Dutch hospital admissions one or more AEs occurred of which 39.6% were preventable. Of all AEs, 12.6% resulted in permanent disability or contributed to death. More than half of the AEs were related to surgical procedures; the proportion of (preventable) AEs and their impact increased with age. Among deceased patients, 10.7% had experienced an AE, and in 4.1% a preventable AE contributed to death, amounting to 1735 potentially preventable hospital deaths in the Netherlands in 2004.
More AEs, but fewer preventable AEs, occurred in university hospitals than in tertiary teaching and general hospitals. Tertiary teaching hospitals in the Netherlands provide highly specialised care and train doctors in collaboration with university hospitals. The level of care given is between that given in a university hospital and in a general hospital. Generally, university hospitals and to some extent tertiary teaching hospitals treat more complex patients with more complex care.17 This may explain the higher incidence rate of AEs with lower preventability in university hospitals. The higher proportion of AEs among older patients may be explained primarily by the clinical complexity of their care rather than an age-based discrimination.18
The incidence of AEs in the Netherlands is substantial, although at the lower end of the range of results from previous retrospective patient record studies; lower incidence rates were reported only in the USA (table 6). However, these studies assessed negligence rather than preventability, which on the one hand may have led to a more defensive assessment. The conservative level for causation (⩾4) may have resulted in lower incidence estimates than in studies using level 2 as a threshold.451314 On the other hand, we only included patient records for which both the nursing and the medical records were present. This may have led to higher estimates, because in almost 90% of our patient records with screening criteria indicating potential AEs, valuable information was detected in the nursing record. A further explanation of the lower incidence of AEs (and the rate of potentially preventable deaths) in our study could be that the review process with two independent doctor reviewers per record has led to a stricter assessment.
Our study has several limitations. We retrospectively determined the potentially preventable deaths and life expectancy of deceased patients in case the AE had not occurred, based on information in hospital records. It is difficult to estimate the probability of death given that the error was not made.6 The reviewers could estimate the life expectancy in 81% of the cases, but the results must be treated with caution. In addition, moderate reliability of the review process is a well-known problem of record review studies to identify AEs and their preventability, in which κ values ranged from 0.2 to 0.6.158132021 We aimed to improve the reliability of the review process by:
intensifying the training of reviewers;
including paediatricians and neurologists as reviewers in addition to general internists and surgeons;
using two doctors instead of a single doctor reviewer;
continuous availability of expert consultation from 18 medical specialties;
frequently updating and communicating a frequently asked questions list for reviewers;
by using electronic review forms.16
However, the level of agreement (κ value) between two pairs of doctors in this study was not better than in other studies. The high number of reviewers and the high proportion of often complex cases of hospital deaths may have led to lower reliability. Another general weakness of all retrospective studies is hindsight bias.22 Knowing the outcome and its severity may influence judgement of causation and preventability. In the present study, this could have led to an overestimation of preventable AEs that contributed to the patient’s death, as judged by the reviewers.
Although judgement of presence of AEs is difficult, retrospective patient record studies are currently the best method available to assess incidence of AEs.23 The results provide urgently needed insight in the current state of patient safety and possibilities for improvement of patient safety and are therefore generally highly appreciated.
Before 2004, there was no widespread public awareness of patient safety in the Netherlands. The results of this study have provided the basis for a patient safety action campaign for hospitals “Prevent harm, work safely”, which started in 2008 and will run until 2012.24
Sufficiently powered studies, or analyses of pooled data from comparable international studies, are needed to get more insight in the incidence of (preventable) AEs for specific specialties or patient groups in order to develop more effective interventions to improve patient safety. An approach that combines record review with prospective methods, in which clinical staff are interviewed about the origin of the AE, will give a more complete picture of (preventable) AEs and organisational and human causal factors.825 To prevent highly preventable AEs, such as AEs related to the diagnostic process, interventions to optimise healthcare procedures and multidisciplinary management are of special interest. AEs currently defined as unpreventable require research to develop new techniques to make them preventable in the future.
We would like to thank everyone who contributed to the study: the nurses and doctors who reviewed the patient records, E van Triest for the coordination of the data collection, the 21 participating hospitals and their staff who facilitated the patient records, and P Spreeuwenberg for statistical assistance. The instruments and protocol originated from the Canadian Adverse Events study. We thank G Ross Baker for his advice.
The calculation of the weighting proportions accounts for the sampling strategy. The sampling weight was the inverse of the probability of being included in the sample owing to the sampling design. It accounts for the proportion of hospital admissions in the three hospital types and the proportion of discharged and deceased patients in the Dutch population of hospital admissions compared with the proportion in this study.
Additional appendices are published online only at http://qshc.bmj.com/content/vol18/issue4
Funding: The Dutch Patient Safety Research Program was initiated by the Orde van Medisch Specialisten [Dutch Society of Medical Specialists] and the Dutch Institute for Healthcare Improvement (CBO) with financial support from the Ministry of Health, Welfare and Sport. The programme is being carried out by EMGO Institute/VUmc and NIVEL.
Competing interests: None.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.