Article Text

PDF

Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study
  1. M Zegers1,
  2. M C de Bruijne2,
  3. C Wagner1,2,
  4. L H F Hoonhout2,
  5. R Waaijman2,
  6. M Smits1,
  7. F A G Hout2,
  8. L Zwaan2,
  9. I Christiaans-Dingelhoff2,
  10. D R M Timmermans2,
  11. P P Groenewegen1,
  12. G van der Wal2
  1. 1
    NIVEL, Netherlands Institute for Health Services Research, Utrecht, the Netherlands
  2. 2
    EMGO Institute, VU University Medical Centre, Amsterdam, the Netherlands
  1. M Zegers, NIVEL, Netherlands Institute for Health Services Research, PO Box 1568, 3500 BN Utrecht, the Netherlands; m.zegers{at}nivel.nl

Abstract

Objective: This study determined the incidence, type, nature, preventability and impact of adverse events (AEs) among hospitalised patients and potentially preventable deaths in Dutch hospitals.

Methods: Using a three-stage retrospective record review process, trained nurses and doctors reviewed 7926 admissions: 3983 admissions of deceased hospital patients and 3943 admissions of discharged patients in 2004, in a random sample of 21 hospitals in the Netherlands (4 university, 6 tertiary teaching and 11 general hospitals). A large sample of deceased patients was included to determine the occurrence of potentially preventable deaths in hospitals more precisely.

Results: One or more AEs were found in 5.7% (95% CI 5.1% to 6.4%) of all admissions and a preventable AE in 2.3% (95% CI 1.9% to 2.7%). Of all AEs, 12.8% resulted in permanent disability or contributed to death. The proportion of AEs and their impact increased with age. More than 50% of the AEs were related to surgical procedures. Among deceased hospital patients, 10.7% (95% CI 9.8% to 11.7%) had experienced an AE. Preventable AEs that contributed to death occurred in 4.1% (95% CI 3.5% to 4.8%) of all hospital deaths. Extrapolating to a national level, between 1482 and 2032 potentially preventable deaths occurred in Dutch hospitals in 2004.

Conclusions: The incidence of AEs, preventable AEs and potentially preventable deaths in the Netherlands is substantial and needs to be reduced. Patient safety efforts should focus on surgical procedures and older patients.

Statistics from Altmetric.com

Previous retrospective record review studies in several countries have shown that 2.9% to 16.6% of patients in acute care hospitals experience one or more adverse events (AEs) and in 4.5–20.8% of the AEs, the patient dies.113 Approximately 50% of the AEs are judged to be preventable. An AE is defined as an unintended injury that results in temporary or permanent disability, death or prolonged hospital stay, and is caused by healthcare management rather than by the patient’s underlying disease process.25813 Variation in the incidence of AEs will be partly determined by policies and characteristics at hospital level (within country variation, between hospitals), partly by policies at national level (between country variation) and partly by differences in methodology of the studies.1415

The occurrence of AEs in Dutch hospitals has not been systematically studied. Insight in preventable AEs can increase the sense of urgency and offer a starting point for specific interventions to improve patient safety, whereas insight in unpreventable AEs may help prioritise research areas. Therefore, we conducted a retrospective patient record review study to determine the incidence, type, nature, impact and preventability of AEs among hospitalised patients in the Netherlands. This study is the largest population-based study carried out in Europe on the occurrence of AEs and reflects the increasing attention paid to patient safety across Europe. The protocol and instruments were based on the Canadian Adverse Events Study.2

METHODS

Box 1 Definitions and outcome measures16

Adverse event determination

AE determination was based on the presence of three criteria:

  1. an unintended (physical and/or mental) injury, which

  2. resulted in temporary or permanent disability, death or prolongation of hospital stay, and was

  3. caused by healthcare management rather than the patient’s disease.

To determine whether the injury was caused by healthcare management or the disease process (criterion 3) a six-point scale was used:

  1. (Virtually) no evidence for management causation

  2. Slight to modest evidence of management causation

  3. Management causation not likely (less than 50/50, but “close call”)

  4. Management causation more likely (more than 50/50, but “close call”)

  5. Moderate to strong evidence of management causation

  6. (Virtually) certain evidence of management causation

Causation scores of 4–6 were classified as AEs.

Timing of AEs

The index hospital admission was the admission sampled. AEs were included if they occurred during the index admission and were detected during or within 12 months after the index admission. AEs were also included if they were related to hospital admissions in the same hospital in the 12 months preceding the index admission, but were not detected until the index admission.

Preventability

The degree of preventability of AEs was measured on a six-point scale, grouped into three categories:

  • No preventability

1. (Virtually) no evidence for preventability

  • Low preventability

2. Slight to modest evidence of preventability

3. Preventability not quite likely (less than 50/50, but “close call”)

  • High preventability

4. Preventability more than likely (more than 50/50, but “close call”)

5. Strong evidence of preventability

6. (Virtually) certain evidence of preventability

AEs with a preventability score of 4–6 were defined as preventable AEs.

Potentially preventable hospital deaths

Potentially preventable hospital deaths were defined as highly preventable AEs which contributed to death during the hospital admission. The adjective “potentially” is used because of the multifactorial nature of hospital deaths and the retrospective assessment of causality.

Life expectancy

The doctor reviewers estimated the life expectancy of the deceased patients for the situation that the hospital admission would have evolved without an AE, taking into account the health status of the patient as described in the patient records and using their professional judgement.

Study design and setting

The design and methods have been described in detail elsewhere.16 We performed a retrospective patient record review study in a random stratified sample of 21 of the 101 Dutch hospitals: 4 university, 6 tertiary teaching and 11 general hospitals. To measure the difference in incidence between hospital types, the sample of hospitals was stratified for hospital type. Proper representation of urban and rural settings in the sample was verified. Eligible hospitals had at least 200 beds, an emergency department and an intensive care unit. A large subsample of deceased hospital patients was included to determine more precisely than in previous studies the occurrence of potentially preventable deaths in hospitals.

The power calculation of this study was based on the results of the Canadian Adverse Events Study.2 Assuming an incidence of AEs of 8%, we required a sample of 4200 hospital admissions of discharged patients and a sample of 4200 admissions of deceased patients (β = 0.20, α = 0.05) to estimate a 95% confidence interval of 0.5% on both sides. To measure the difference in incidence between hospital types, a selection of 800 hospital admissions per hospital type was necessary to detect a difference from 2% to 3% by an incidence between 3% and 7%.

From each hospital, we randomly selected 200 admissions (>24 h stay) of discharged patients and 200 (or fewer if the total of patients who died in 2004 was lower) admissions of deceased hospital patients in 2004, excluding admissions to psychiatry and obstetrics and of children <1 year.16

Review of patient records

The nursing, medical and, if available, outpatient record of the sampled admissions were reviewed by 66 trained nurses and 55 trained doctors in a three-stage review process between August 2005 and October 2006. In the first stage, a nurse screened the records by using 18 screening criteria indicating potential AEs (see online appendix A). In the second stage, two doctors independently reviewed the records with one or more positive screening criteria. Based on a standardised procedure they determined presence, nature, impact, clinical process and degree of preventability of the AEs. Also the life expectancy in case the AE had not occurred was estimated for deceased patients with AEs (box 1). If there was disagreement about the presence and/or preventability of an AE between the two doctors’ reviews, they undertook a consensus procedure (stage 3). If they could not reach a consensus, a third trained doctor reviewer gave the final judgement.16

Reliability study

To assess the reliability of the first stage of the record review process, a random sample of 415 records was independently reviewed by a second nurse. To assess the reliability of the final judgement (including any consensus and third review if applicable) of the doctors’ review, a random sample of 119 records was reviewed by a second pair of doctors.16

Statistical analysis

We calculated the national weighted incidence of AEs and preventable AEs in Dutch hospitals with 95% confidence intervals, corrected for the over-representation of patients admitted to a university hospital and for the over-representation of patients who died in hospital, using the “complex sample” option in SPSS V14.0. The sample weight was the inverse of the probability of being included in the sample owing to the sample design. It was calculated as N/n, where N =  the number of elements in the population and n =  the number of elements in the sample (see online appendix B). After weighting for the sampling frame, the total study sample (discharged and deceased patients) and the subsample of deceased patients were representative for the Dutch population of hospitalised patients and for the Dutch population of deceased hospital patients, respectively (see online appendix C). The characteristics of the AEs, such as disability and classification, and patient and admission characteristics were also assessed using weights to adjust for the sampling frame. The analysis for the subsample of deceased patients was only weighted for the over-representation of patients admitted to a university hospital.

The inter-rater agreement of the review process between nurses for finding screening criteria and for the determination of AEs and their preventability between two pairs of doctors was expressed as a κ statistic with 95% confidence intervals and as a percentage of records for which there was agreement.

RESULTS

Incidence of AEs among hospitalised patients

Of the 8415 sampled records, 8032 were eligible for a first-stage review (screening success rate was 95%) (fig 1). In 54% of these records one or more screening criteria were identified (appendix A). In the second stage, the reviewing doctors identified 744 AEs in 663 hospital admissions. In 70 hospital admissions (10.6% of 663), two or more AEs were found. In all, 504 AEs were found in 447 deceased and 240 AEs in 216 discharged patients. After weighing for the sampling frame, the national incidence of AEs among hospitalised patients was 5.7% (95% CI 5.1% to 6.4%) and the incidence of preventable AEs was 2.3% (95% CI 1.9% to 2.7%) (table 1).

Figure 1

Flowchart of the review process. *Records were considered not available when the nursing and/or medical record of the index admission was missing or because the patient was hospitalised during the study. For one hospital, only the records of hospital deaths were included because of hospital-related logistical reasons. Records were considered inadequate if they: did not comply with the selection criteria; contained inadequate and/or incomplete documentation for AE determination; were sampled twice or wrongly; or the admission was too short (patient came with cardiac arrest to the emergency department or outpatient resuscitation). †Patient was hospitalised during the study; inadequate and/or incomplete patient record documentation; did not comply with the selection criteria; sampled twice; incomplete record review.

Table 1 Weighted incidence of adverse events (AEs) in patients admitted to Dutch hospitals in 2004 by hospital type, both for the total sample (discharged patients and deceased patients) and for the subgroup of deceased patients

Incidence rates of AEs were significantly higher in university hospitals than in general hospitals (p<0.001). Although not statistically significant, the incidence of preventable AEs was lower in university hospitals compared with the other types of hospital.

Of 744 AEs, 39.6% were considered preventable. More than half of all AEs (56.8%) resulted in no or minimal physical impairment or disability. However, 5.0% resulted in permanent disability and 7.8% contributed to death. The incidence of AEs was higher in surgical departments than in non-surgical departments (table 2).

Table 2 Adverse events (AEs) by admission department: number of reviewed admissions, weighted AE incidence per department and proportion of AEs that were preventable

A quarter of all AEs occurred in the previous 12 months and were detected in the index admission in the same hospital. The majority of AEs (63%) occurred and were detected during the index admission, whereas 12% occurred during the index admission and were detected within 12 months after the index admission.

More than half (54%) of all AEs were related to surgical procedures. Almost all AEs related to the diagnostic process were highly preventable and 23% contributed to death (table 3). The proportion of AEs, preventable AEs and degree of disability increased with age (table 4).

Table 3 Adverse events (AEs) by clinical process and proportions judged preventable, leading to permanent disability (excluding death) and contributed to death
Table 4 Adverse events (AEs) by age: number of reviewed admissions, number of admissions with an AE and proportions judged preventable, contributed to permanent disability (excluding death) and death

Incidence of AEs among deceased patients

The incidence of AEs and preventable AEs in the subsample of deceased patients was 10.7% (95% CI 9.8% to 11.7%) and 5.2% (95% CI 4.5% to 5.9%), respectively (table 1). The incidence of preventable AEs contributing to death among deceased patients was 4.1% (95% CI 3.5% to 4.8%). In 2004, 42 329 patients died in Dutch hospitals, amounting to 1735 (95% CI 1482 to 2032) potentially preventable hospital deaths.

Compared with the total hospital population, AEs among deceased patients were more often preventable (47.7% vs 39.6%) and were more often related to the diagnostic process (14.8% vs 6.3%). Almost half (49.1%) of the deceased patients with a potentially preventable AE that contributed to death had an estimated potential life expectancy of more than 1 year had the potentially preventable AE not have occurred. In 11% of the cases the life expectancy could not be determined (table 5).

Table 5 Estimated life expectancy of deceased patients with adverse events (AEs), preventable AEs and potentially preventable AEs that were associated with death

Agreement between reviewers

The reliability of the assessment of screening criteria by nurses was good (κ 0.62; 95% CI 0.54 to 0.69; 82% agreement). The reliability of determination of AEs was only fair (κ 0.25; 95% CI 0.05 to 0.45; 76% agreement), and it was moderate for determination of preventability of AEs (κ 0.40; 95% CI 0.07 to 0.73; 70% agreement).

DISCUSSION

The present study found that in 5.7% of all Dutch hospital admissions one or more AEs occurred of which 39.6% were preventable. Of all AEs, 12.6% resulted in permanent disability or contributed to death. More than half of the AEs were related to surgical procedures; the proportion of (preventable) AEs and their impact increased with age. Among deceased patients, 10.7% had experienced an AE, and in 4.1% a preventable AE contributed to death, amounting to 1735 potentially preventable hospital deaths in the Netherlands in 2004.

More AEs, but fewer preventable AEs, occurred in university hospitals than in tertiary teaching and general hospitals. Tertiary teaching hospitals in the Netherlands provide highly specialised care and train doctors in collaboration with university hospitals. The level of care given is between that given in a university hospital and in a general hospital. Generally, university hospitals and to some extent tertiary teaching hospitals treat more complex patients with more complex care.17 This may explain the higher incidence rate of AEs with lower preventability in university hospitals. The higher proportion of AEs among older patients may be explained primarily by the clinical complexity of their care rather than an age-based discrimination.18

The incidence of AEs in the Netherlands is substantial, although at the lower end of the range of results from previous retrospective patient record studies; lower incidence rates were reported only in the USA (table 6). However, these studies assessed negligence rather than preventability, which on the one hand may have led to a more defensive assessment. The conservative level for causation (⩾4) may have resulted in lower incidence estimates than in studies using level 2 as a threshold.451314 On the other hand, we only included patient records for which both the nursing and the medical records were present. This may have led to higher estimates, because in almost 90% of our patient records with screening criteria indicating potential AEs, valuable information was detected in the nursing record. A further explanation of the lower incidence of AEs (and the rate of potentially preventable deaths) in our study could be that the review process with two independent doctor reviewers per record has led to a stricter assessment.

Table 6 Retrospective record review studies on the occurrence of (preventable) AEs of hospitalised patients between 1984 and 2005

Our study has several limitations. We retrospectively determined the potentially preventable deaths and life expectancy of deceased patients in case the AE had not occurred, based on information in hospital records. It is difficult to estimate the probability of death given that the error was not made.6 The reviewers could estimate the life expectancy in 81% of the cases, but the results must be treated with caution. In addition, moderate reliability of the review process is a well-known problem of record review studies to identify AEs and their preventability, in which κ values ranged from 0.2 to 0.6.158132021 We aimed to improve the reliability of the review process by:

  • intensifying the training of reviewers;

  • including paediatricians and neurologists as reviewers in addition to general internists and surgeons;

  • using two doctors instead of a single doctor reviewer;

  • continuous availability of expert consultation from 18 medical specialties;

  • frequently updating and communicating a frequently asked questions list for reviewers;

  • by using electronic review forms.16

However, the level of agreement (κ value) between two pairs of doctors in this study was not better than in other studies. The high number of reviewers and the high proportion of often complex cases of hospital deaths may have led to lower reliability. Another general weakness of all retrospective studies is hindsight bias.22 Knowing the outcome and its severity may influence judgement of causation and preventability. In the present study, this could have led to an overestimation of preventable AEs that contributed to the patient’s death, as judged by the reviewers.

Although judgement of presence of AEs is difficult, retrospective patient record studies are currently the best method available to assess incidence of AEs.23 The results provide urgently needed insight in the current state of patient safety and possibilities for improvement of patient safety and are therefore generally highly appreciated.

Before 2004, there was no widespread public awareness of patient safety in the Netherlands. The results of this study have provided the basis for a patient safety action campaign for hospitals “Prevent harm, work safely”, which started in 2008 and will run until 2012.24

Sufficiently powered studies, or analyses of pooled data from comparable international studies, are needed to get more insight in the incidence of (preventable) AEs for specific specialties or patient groups in order to develop more effective interventions to improve patient safety. An approach that combines record review with prospective methods, in which clinical staff are interviewed about the origin of the AE, will give a more complete picture of (preventable) AEs and organisational and human causal factors.825 To prevent highly preventable AEs, such as AEs related to the diagnostic process, interventions to optimise healthcare procedures and multidisciplinary management are of special interest. AEs currently defined as unpreventable require research to develop new techniques to make them preventable in the future.

Acknowledgments

We would like to thank everyone who contributed to the study: the nurses and doctors who reviewed the patient records, E van Triest for the coordination of the data collection, the 21 participating hospitals and their staff who facilitated the patient records, and P Spreeuwenberg for statistical assistance. The instruments and protocol originated from the Canadian Adverse Events study. We thank G Ross Baker for his advice.

APPENDIX

Appendix A

Table A1 Screening criteria applied in stage 1 to 3983 records of deceased patients and 3943 records of discharged patients and the proportion of hospital admissions positive for each criterion*

Appendix B

Weighting proportions

The calculation of the weighting proportions accounts for the sampling strategy. The sampling weight was the inverse of the probability of being included in the sample owing to the sampling design. It accounts for the proportion of hospital admissions in the three hospital types and the proportion of discharged and deceased patients in the Dutch population of hospital admissions compared with the proportion in this study.

Table B1 Proportions of hospital admissions for discharged status and hospital type
Table B2 Weighting proportions for the different sampling strata

Appendix C

Table C1 Comparison of study patients and patients admitted to all Dutch acute care hospitals in 2004

REFERENCES

View Abstract

Footnotes

  • Additional appendices are published online only at http://qshc.bmj.com/content/vol18/issue4

  • Funding: The Dutch Patient Safety Research Program was initiated by the Orde van Medisch Specialisten [Dutch Society of Medical Specialists] and the Dutch Institute for Healthcare Improvement (CBO) with financial support from the Ministry of Health, Welfare and Sport. The programme is being carried out by EMGO Institute/VUmc and NIVEL.

  • Competing interests: None.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.