Previous retrospective record review studies in several countries have shown that 2.9% to 16.6% of patients in acute care hospitals experience one or more adverse events (AEs) and in 4.5–20.8% of the AEs, the patient dies.1^–13 Approximately 50% of the AEs are judged to be preventable. An AE is defined as an unintended injury that results in temporary or permanent disability, death or prolonged hospital stay, and is caused by healthcare management rather than by the patient’s underlying disease process.25813 Variation in the incidence of AEs will be partly determined by policies and characteristics at hospital level (within country variation, between hospitals), partly by policies at national level (between country variation) and partly by differences in methodology of the studies.1415

The occurrence of AEs in Dutch hospitals has not been systematically studied. Insight in preventable AEs can increase the sense of urgency and offer a starting point for specific interventions to improve patient safety, whereas insight in unpreventable AEs may help prioritise research areas. Therefore, we conducted a retrospective patient record review study to determine the incidence, type, nature, impact and preventability of AEs among hospitalised patients in the Netherlands. This study is the largest population-based study carried out in Europe on the occurrence of AEs and reflects the increasing attention paid to patient safety across Europe. The protocol and instruments were based on the Canadian Adverse Events Study.2

METHODS

Study design and setting

The design and methods have been described in detail elsewhere.16 We performed a retrospective patient record review study in a random stratified sample of 21 of the 101 Dutch hospitals: 4 university, 6 tertiary teaching and 11 general hospitals. To measure the difference in incidence between hospital types, the sample of hospitals was stratified for hospital type. Proper representation of urban and rural settings in the sample was verified. Eligible hospitals had at least 200 beds, an emergency department and an intensive care unit. A large subsample of deceased hospital patients was included to determine more precisely than in previous studies the occurrence of potentially preventable deaths in hospitals.

The power calculation of this study was based on the results of the Canadian Adverse Events Study.2 Assuming an incidence of AEs of 8%, we required a sample of 4200 hospital admissions of discharged patients and a sample of 4200 admissions of deceased patients (β = 0.20, α = 0.05) to estimate a 95% confidence interval of 0.5% on both sides. To measure the difference in incidence between hospital types, a selection of 800 hospital admissions per hospital type was necessary to detect a difference from 2% to 3% by an incidence between 3% and 7%.

From each hospital, we randomly selected 200 admissions (>24 h stay) of discharged patients and 200 (or fewer if the total of patients who died in 2004 was lower) admissions of deceased hospital patients in 2004, excluding admissions to psychiatry and obstetrics and of children <1 year.16

Review of patient records

The nursing, medical and, if available, outpatient record of the sampled admissions were reviewed by 66 trained nurses and 55 trained doctors in a three-stage review process between August 2005 and October 2006. In the first stage, a nurse screened the records by using 18 screening criteria indicating potential AEs (see online appendix A). In the second stage, two doctors independently reviewed the records with one or more positive screening criteria. Based on a standardised procedure they determined presence, nature, impact, clinical process and degree of preventability of the AEs. Also the life expectancy in case the AE had not occurred was estimated for deceased patients with AEs (box 1). If there was disagreement about the presence and/or preventability of an AE between the two doctors’ reviews, they undertook a consensus procedure (stage 3). If they could not reach a consensus, a third trained doctor reviewer gave the final judgement.16

Reliability study

To assess the reliability of the first stage of the record review process, a random sample of 415 records was independently reviewed by a second nurse. To assess the reliability of the final judgement (including any consensus and third review if applicable) of the doctors’ review, a random sample of 119 records was reviewed by a second pair of doctors.16

Statistical analysis

We calculated the national weighted incidence of AEs and preventable AEs in Dutch hospitals with 95% confidence intervals, corrected for the over-representation of patients admitted to a university hospital and for the over-representation of patients who died in hospital, using the “complex sample” option in SPSS V14.0. The sample weight was the inverse of the probability of being included in the sample owing to the sample design. It was calculated as N/n, where N =  the number of elements in the population and n =  the number of elements in the sample (see online appendix B). After weighting for the sampling frame, the total study sample (discharged and deceased patients) and the subsample of deceased patients were representative for the Dutch population of hospitalised patients and for the Dutch population of deceased hospital patients, respectively (see online appendix C). The characteristics of the AEs, such as disability and classification, and patient and admission characteristics were also assessed using weights to adjust for the sampling frame. The analysis for the subsample of deceased patients was only weighted for the over-representation of patients admitted to a university hospital.

The inter-rater agreement of the review process between nurses for finding screening criteria and for the determination of AEs and their preventability between two pairs of doctors was expressed as a κ statistic with 95% confidence intervals and as a percentage of records for which there was agreement.

RESULTS

Incidence of AEs among hospitalised patients

Of the 8415 sampled records, 8032 were eligible for a first-stage review (screening success rate was 95%) (fig 1). In 54% of these records one or more screening criteria were identified (appendix A). In the second stage, the reviewing doctors identified 744 AEs in 663 hospital admissions. In 70 hospital admissions (10.6% of 663), two or more AEs were found. In all, 504 AEs were found in 447 deceased and 240 AEs in 216 discharged patients. After weighing for the sampling frame, the national incidence of AEs among hospitalised patients was 5.7% (95% CI 5.1% to 6.4%) and the incidence of preventable AEs was 2.3% (95% CI 1.9% to 2.7%) (table 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Flowchart of the review process. *Records were considered not available when the nursing and/or medical record of the index admission was missing or because the patient was hospitalised during the study. For one hospital, only the records of hospital deaths were included because of hospital-related logistical reasons. Records were considered inadequate if they: did not comply with the selection criteria; contained inadequate and/or incomplete documentation for AE determination; were sampled twice or wrongly; or the admission was too short (patient came with cardiac arrest to the emergency department or outpatient resuscitation). †Patient was hospitalised during the study; inadequate and/or incomplete patient record documentation; did not comply with the selection criteria; sampled twice; incomplete record review.

Incidence rates of AEs were significantly higher in university hospitals than in general hospitals (p<0.001). Although not statistically significant, the incidence of preventable AEs was lower in university hospitals compared with the other types of hospital.

Of 744 AEs, 39.6% were considered preventable. More than half of all AEs (56.8%) resulted in no or minimal physical impairment or disability. However, 5.0% resulted in permanent disability and 7.8% contributed to death. The incidence of AEs was higher in surgical departments than in non-surgical departments (table 2).

A quarter of all AEs occurred in the previous 12 months and were detected in the index admission in the same hospital. The majority of AEs (63%) occurred and were detected during the index admission, whereas 12% occurred during the index admission and were detected within 12 months after the index admission.

More than half (54%) of all AEs were related to surgical procedures. Almost all AEs related to the diagnostic process were highly preventable and 23% contributed to death (table 3). The proportion of AEs, preventable AEs and degree of disability increased with age (table 4).

Incidence of AEs among deceased patients

The incidence of AEs and preventable AEs in the subsample of deceased patients was 10.7% (95% CI 9.8% to 11.7%) and 5.2% (95% CI 4.5% to 5.9%), respectively (table 1). The incidence of preventable AEs contributing to death among deceased patients was 4.1% (95% CI 3.5% to 4.8%). In 2004, 42 329 patients died in Dutch hospitals, amounting to 1735 (95% CI 1482 to 2032) potentially preventable hospital deaths.

Compared with the total hospital population, AEs among deceased patients were more often preventable (47.7% vs 39.6%) and were more often related to the diagnostic process (14.8% vs 6.3%). Almost half (49.1%) of the deceased patients with a potentially preventable AE that contributed to death had an estimated potential life expectancy of more than 1 year had the potentially preventable AE not have occurred. In 11% of the cases the life expectancy could not be determined (table 5).

Agreement between reviewers

The reliability of the assessment of screening criteria by nurses was good (κ 0.62; 95% CI 0.54 to 0.69; 82% agreement). The reliability of determination of AEs was only fair (κ 0.25; 95% CI 0.05 to 0.45; 76% agreement), and it was moderate for determination of preventability of AEs (κ 0.40; 95% CI 0.07 to 0.73; 70% agreement).

DISCUSSION

The present study found that in 5.7% of all Dutch hospital admissions one or more AEs occurred of which 39.6% were preventable. Of all AEs, 12.6% resulted in permanent disability or contributed to death. More than half of the AEs were related to surgical procedures; the proportion of (preventable) AEs and their impact increased with age. Among deceased patients, 10.7% had experienced an AE, and in 4.1% a preventable AE contributed to death, amounting to 1735 potentially preventable hospital deaths in the Netherlands in 2004.

More AEs, but fewer preventable AEs, occurred in university hospitals than in tertiary teaching and general hospitals. Tertiary teaching hospitals in the Netherlands provide highly specialised care and train doctors in collaboration with university hospitals. The level of care given is between that given in a university hospital and in a general hospital. Generally, university hospitals and to some extent tertiary teaching hospitals treat more complex patients with more complex care.17 This may explain the higher incidence rate of AEs with lower preventability in university hospitals. The higher proportion of AEs among older patients may be explained primarily by the clinical complexity of their care rather than an age-based discrimination.18

The incidence of AEs in the Netherlands is substantial, although at the lower end of the range of results from previous retrospective patient record studies; lower incidence rates were reported only in the USA (table 6). However, these studies assessed negligence rather than preventability, which on the one hand may have led to a more defensive assessment. The conservative level for causation (⩾4) may have resulted in lower incidence estimates than in studies using level 2 as a threshold.451314 On the other hand, we only included patient records for which both the nursing and the medical records were present. This may have led to higher estimates, because in almost 90% of our patient records with screening criteria indicating potential AEs, valuable information was detected in the nursing record. A further explanation of the lower incidence of AEs (and the rate of potentially preventable deaths) in our study could be that the review process with two independent doctor reviewers per record has led to a stricter assessment.

Our study has several limitations. We retrospectively determined the potentially preventable deaths and life expectancy of deceased patients in case the AE had not occurred, based on information in hospital records. It is difficult to estimate the probability of death given that the error was not made.6 The reviewers could estimate the life expectancy in 81% of the cases, but the results must be treated with caution. In addition, moderate reliability of the review process is a well-known problem of record review studies to identify AEs and their preventability, in which κ values ranged from 0.2 to 0.6.158132021 We aimed to improve the reliability of the review process by:

• intensifying the training of reviewers;

• including paediatricians and neurologists as reviewers in addition to general internists and surgeons;

• using two doctors instead of a single doctor reviewer;

• continuous availability of expert consultation from 18 medical specialties;

• frequently updating and communicating a frequently asked questions list for reviewers;

• by using electronic review forms.16

However, the level of agreement (κ value) between two pairs of doctors in this study was not better than in other studies. The high number of reviewers and the high proportion of often complex cases of hospital deaths may have led to lower reliability. Another general weakness of all retrospective studies is hindsight bias.22 Knowing the outcome and its severity may influence judgement of causation and preventability. In the present study, this could have led to an overestimation of preventable AEs that contributed to the patient’s death, as judged by the reviewers.

Although judgement of presence of AEs is difficult, retrospective patient record studies are currently the best method available to assess incidence of AEs.23 The results provide urgently needed insight in the current state of patient safety and possibilities for improvement of patient safety and are therefore generally highly appreciated.

Before 2004, there was no widespread public awareness of patient safety in the Netherlands. The results of this study have provided the basis for a patient safety action campaign for hospitals “Prevent harm, work safely”, which started in 2008 and will run until 2012.24

Sufficiently powered studies, or analyses of pooled data from comparable international studies, are needed to get more insight in the incidence of (preventable) AEs for specific specialties or patient groups in order to develop more effective interventions to improve patient safety. An approach that combines record review with prospective methods, in which clinical staff are interviewed about the origin of the AE, will give a more complete picture of (preventable) AEs and organisational and human causal factors.825 To prevent highly preventable AEs, such as AEs related to the diagnostic process, interventions to optimise healthcare procedures and multidisciplinary management are of special interest. AEs currently defined as unpreventable require research to develop new techniques to make them preventable in the future.