You are here

Article Text

Article menu

Download PDFPDF

Original research
Changes in adverse event rates in hospitals over time: a longitudinal retrospective patient record review study
Free
  1. Rebecca J Baines1,
  2. Maaike Langelaan2,
  3. Martine C de Bruijne1,
  4. Henk Asscheman3,
  5. Peter Spreeuwenberg2,
  6. Lotte van de Steeg2,
  7. Kitty M Siemerink2,
  8. Floor van Rosse1,4,
  9. Maren Broekens1,
  10. Cordula Wagner1,2
  1. 1Department of Public and Occupational Health, EMGO+ Institute for Health and Care Research, VU University Medical Center (VUmc), Amsterdam, The Netherlands
  2. 2NIVEL, Netherlands Institute for Health Services Research, Utrecht, The Netherlands
  3. 3HAJAP, consultant internal medicine, Amsterdam, The Netherlands
  4. 4Department of Public Health, Academic Medical Center (AMC), Amsterdam, The Netherlands
  1. Correspondence to Rebecca J Baines, Department of Public and Occupation Health,EMGO+ Institute/VU University Medical Center, Van der Boechorststraat 7, room B-550, Amsterdam 1081 BT, The Netherlands; r.baines{at}vumc.nl

Abstract

Objective To determine the change in adverse event (AE) rates and preventable AE rates over time, identify certain patient risk groups and discuss factors influencing the outcome.

Design Longitudinal retrospective patient record review study.

Setting and participants A random sample of 21 hospitals in The Netherlands in 2004, and 20 hospitals in 2008. In each hospital, 400 patient admissions were included in 2004, and 200 in 2008.

Main outcome measures AEs and preventable AEs.

Results Multilevel analyses of 11 883 patient records (7.887 in 2004, 3.996 in 2008) showed that the rate of patients experiencing an AE increased from 4.1% (95% CI 3.3% to 5.1%) in 2004 to 6.2% (95% CI 5.0% to 7.6%) in 2008. The preventable AE rate remained relatively stable at 1.8% (95% CI 1.3% to 2.4%) in 2004 and 1.6% (95% CI 1.2% to 2.3%) in 2008. The risk of experiencing a preventable AE was increasingly higher for patients admitted to a surgical unit (OR 1.54 (95% CI 1.10 to 2.16) in 2004 and 3.32 (95% CI 2.17 to 5.07)) in 2008. More than 50% of all AEs were related to surgery. Indications were found that differences in the risk of experiencing a preventable AE between hospital departments were larger in 2008 than in 2004, while differences between hospitals themselves were smaller.

Conclusions Patient harm related to healthcare is a persistent problem that is hard to influence. Measuring AEs over time stresses the continuing urgency, and also identifies possible areas for improvement.

  • Medical error, measurement/epidemiology
  • Adverse events, epidemiology and detection
  • Patient safety
  • Hospital medicine
  • Health policy

https://doi.org/10.1136/bmjqs-2012-001126

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Following the Harvard Medical Practice Study (HMPS),1 many countries have published their own results on adverse event (AE) rates.2–9 In response, patient safety interventions and campaigns have been set up to improve patient outcomes by reducing harm related to healthcare. Evidence of their effect is, however, limited.10–12 Long-term change in AE and preventable AE rates could provide an insight to overall development of patient safety in hospitals.

    Measuring AEs over time on a large scale is important to estimate the overall effect of many efforts to improve patient safety, but is also difficult. It is costly and not all contextual factors that are of influence can be accounted for. Landrigan and colleagues published trends in harm rates in 10 hospitals in Northern Carolina using the IHI Global Trigger Tool (GTT).13 ,14 They were unable to find a significant improvement in preventable AE rates. The GTT was originally developed to fulfil hospitals’ need for a more practical and less labour-intensive approach to assess patient safety.15

    In The Netherlands, the first national AE study took place in 2004 in 21 hospitals using methods comparable with the HMPS and the Canadian AE study.1 ,5 ,16 This method is more labour intensive than the GTT, however, it does allow the physician reviewers to ascertain a broad and structured perspective of a patient, the reviewed admission and the events preceding and following a possible AE. Due to the structured questioning method, a good assessment can be made of the relationship between harm, healthcare and preventability.

    We have undertaken a second measurement to get a more up-to-date view, and to assess changes in national AE and preventable AE rates between 2004 and 2008. We also aim to identify potential patient risk groups and to discuss factors influencing the outcome. The results will help hospitals and patient safety initiatives to identify further possibilities for improvement.

    Methods

    Design and setting

    We performed a longitudinal retrospective patient record review study in 21 randomly sampled hospitals in 2004, and 20 in 2008 out of the total of 93 Dutch hospitals. Eight hospitals were studied in both years. Both samples were stratified for hospital type, university, tertiary teaching and general hospitals, and a proper representation of both urban and rural settings in the samples were verified. Tertiary teaching hospitals in The Netherlands provide specialised care and train doctors. The level of care given is between that given in a university hospital and in a general hospital. Generally speaking, university hospitals and, to some extent, tertiary teaching hospitals tend to treat more complex patients with more complex care. To be eligible, hospitals had to have at least 200 beds and an intensive care unit. In each hospital, 400 patient admissions were randomly selected in 2004, and 200 in 2008. Fifty percent of the records were of patients who were discharged from the hospital after a stay of at least 24 h. The other 50% were of patients who died in hospital. These patients were sampled from all inpatient deaths, regardless of their length of stay. We did not exclude patients admitted with an explicitly palliative care plan; this information was noted down and taken into account during the review process. During analysis, overall AE rates were corrected for the oversampling of deceased patients, because in our sample, 50% of the patients were inpatient deaths, and in reality 3%. In the results, we weight our 50% back to the actual 3%, so the presented results are a representation of the total hospital population of discharged and deceased patients. We followed the same procedure for the distribution of types of hospitals. Patients admitted to the psychiatry department, obstetrics and children under 1 year of age were excluded. The design and results of the 2004 study have been published elsewhere.2 ,16–21

    Record review

    The nursing, medical and, if available, outpatient records of the sample patient admissions were reviewed by trained external nurses and trained external physicians belonging to the specialties surgery, internal medicine and neurology. Consultation with specialties other than their own was available if needed. For the largest part, the reviewers in the 2008 study had also participated in the 2004 study. Review of the records took place in 2005/2006, and in 2009/2010, respectively.

    The method of determining AEs was comparable with those of other international studies.5 ,8 First, a nurse screened the records by using triggers indicating potential AEs, for instance, readmissions or hospital-acquired infections. Admissions positive for at least one trigger were further reviewed by a physician. Based on a standardised procedure, and preceded by a number of underlying questions to secure a systematic assessment, the presence and preventability of an AE was determined.

    An AE was defined by three criteria:

    1. An unintended injury

    2. The injury resulted in prolongation of hospital stay, temporary or permanent disability or death

    3. The injury was caused by healthcare management rather than the patient's disease.

    An AE was found to be preventable when the care given fell below the current level of expected performance for practitioners or systems. The cause of an AE, as well as its preventability, were scored on a 6-point Likert scale and counted as caused by healthcare, or preventable if the score was 4–6. A score of 4–6 indicated that the reviewer regarded the event as having a greater than 50% chance of being caused by healthcare, or preventable. To add more structure to the implicit review process, the causation and preventability score were each preceded by 13 questions to facilitate the final reviewers’ judgment (Appendix 1).

    Appendix 1

    To add more structure to the implicit review process, the causation and preventability score were each preceded by a number of questions to facilitate the final reviewers’ judgment. Our review form is originally based on the Canadian AE study of Baker et al (2004),5 with minor adjustments to the Dutch situation. Below are the sections of our review form concerning these preparatory questions as well as the final judgement.

    Causation, preparatory questions:

    • Is there a note in the medical record indicating that a healthcare professional, or healthcare management, caused the injury? (No/Yes/Not applicable)

    • Is there a note in the medical record suggesting the possibility of an unintended injury from the patient's disease? (No/Yes/Not applicable)

    • Does the timing of events suggest that the injury is related to the treatment? (Likely/Possibly/Unlikely/Not applicable)

    • Does the timing of events suggest that the injury was related to the lack of treatment? (Likely/Possibly/Unlikely/Not applicable)

    • Are there other reasonable explanations for the cause of the unintended injury? (No/Yes/Possibly/Not applicable)

    • Was there an opportunity prior to the occurrence of the injury for intervention which might have prevented it? (No/Yes/Possibly/Not applicable)

    • Is lack of treatment or delayed treatment a recognised cause of this injury? (Widely recognised/Recognised by other specialists/No/Not applicable)

    • Is the lack of diagnosis or delayed diagnosis a recognised cause of this injury? (Widely recognised/Recognised by other specialists/No/Not applicable)

    • Is the treatment given to the patient a recognised cause of this injury? (Widely recognised/Recognised by other specialists/No/Not applicable)

    • Is this injury a recognised complication of the patient's underlying index disease? (Widely recognised/Recognised by other specialists/No/Not applicable)

    • Was the injury recognised during the index admission? (No/Yes/Not applicable)

    • If ‘Yes’, was the appropriate action taken during the index admission? (No/Yes/No action needed/Not applicable)

    • If ‘Yes’, did the injury respond to the appropriate action? (No/Yes/Possibly/Not applicable)

    Causation, final judgement:

    • After consideration of the clinical details of the patient's management, irrespective of preventability, and your response to the questions above, what level of confidence do you have that the healthcare professional or healthcare management caused the injury?

      • (Virtually) no evidence for healthcare management causation

      • Slight to modest evidence of healthcare management causation

      • Healthcare management causation not likely (less than 50/50, but ‘close call’)

      • Healthcare management causation more likely (more than 50/50, but ‘close call’)

      • Moderate to strong evidence of healthcare management causation

      • (Virtually) certain evidence of healthcare management causation.

    Preventablity, preparatory questions:

    • How complex was this case? (Very complex/Moderately complex/Somewhat complex/Not complex/Unable to determine)

    • Was the management of the primary illness (not the adverse event) appropriate? (Definitely appropriate/Possibly appropriate/Probably appropriate/Definitely not appropriate)

    • What was the degree of deviation of management of the primary illness (not the adverse event) from the accepted norm? (Severe/Moderate/Little/None)

    • What was the comorbidity of the patient? (Significant comorbidity/Moderate comorbidity/Mild comorbidity/No comorbidity)

    • What was the degree of emergency in management of the primary illness (not the adverse event) prior to the occurrence of adverse event? (Very urgent/Moderately urgent/Not urgent)

    • What potential benefit was associated with the management of the illness which led to the Adverse Event? (Life saving/Curing/Life prolonging/Symptom relief/Palliation/No potential benefit)

    • What was the chance of benefit associated with the management of the illness which led to the adverse event? (High/Moderate/Low/Not applicable)

    • What was the risk of an adverse event related to the management? (High/Moderate/Low/Not applicable)

    • Is the injury/complication a recognised complication? (No/Yes/Not applicable)

    • What percentage of patients like this would be expected to have this complication? (Unable to determine (UTD)/Not applicable/<1/1%−9%/10%−24%/>=25%)

    • On reflection, would a reasonable doctor or health professional repeat this healthcare management strategy again? (Definitely/Probably/Probably not/Definitely not

    • Was there a comment in the medical records indicating a need for follow-up as a result of this adverse event? (select all that apply) (No/Counselling/Psychiatric/Rehabilitation/Routine clinical/Other/UTD)

    • Did the patient have any follow-up as a result of this adverse event? (No/Counselling/Psychiatric/ Rehabilitation/Routine clinical/Other/UTD)

    Preventability, final judgement:

    • Please indicate to what extent there are indications that the event was preventable:

      • (Virtually) no evidence for preventability

      • Slight to modest evidence of preventability

      • Preventability not quite likely (less than 50/50, but ‘close call’)

      • Preventability more than likely (more than 50/50, but ‘close call’)

      • Strong evidence of preventability

      • (Virtually) certain evidence of preventability

    AEs that occurred during the patient's index hospital admission, and were detected during either the index admission or subsequent admissions over the following 12-month period were counted. Also counted were AEs related to patient admissions in the same hospital within the 12 months preceding the index admission, but which were not detected until the index admission. Consequently, patient records of the index hospital admission were reviewed, as were the patient records of patient admissions before and after the index admission. The way the AEs were counted was the same for both years. Only patient admissions in participating hospitals were evaluated. Out-of-hospital preadmission care could not be evaluated.

    The review process of the 2004 study was slightly adapted for the 2008 study. In the 2004 study, pairs of physicians independently assessed all records positive for screening criteria in the first-stage review. Disagreement about the presence and/or preventability of an AE prompted a consensus procedure.16 Analysis of the data from 2004 showed that physicians within pairs tended to show substantial agreement, however, between-pairs agreement was much lower.17 The involvement of a second reviewer and consensus procedure in 2004 apparently did not per se improve the overall reliability. For this reason, and due to limited resources, we chose to perform a more efficient review process in the 2008 study, as other recent and earlier patient record review studies have also done, in which all records positive for screening criteria were reviewed by one physician.5 ,22 To compensate for the loss of discussion between physicians during the consensus procedure, for the 2008 study we more often organised reflection meetings based on discrepancies in records reviewed twice for all reviewers to uphold high quality of the review process.

    Additional data on the total Dutch hospital population in 2004 and 2008, and on diagnosis coded according to the International Statistical Classification of Diseases, 9th revision (ICD-9) for the reviewed patient admissions, was obtained from the national hospital administration database (Prismant).

    Reliability

    To test the reliability of the review process, a random sample equally spread over all hospitals and time was drawn in 2004 and 2008. To ascertain the reliability of screening by nurses, 415 records were reviewed twice in 2004, and 238 in 2008. To ascertain the reliability of judgments on the presence and preventability of an AE, 120 records were reviewed twice by physicians in 2004, and 228 in 2008. The second reviewer was blinded for the outcome of the first review. The inter-rater agreement was expressed as the percentage of records for which there was agreement on the presence of screening criteria for the nurses, and the presence and preventability of an AE for the physicians, and by the κ statistic.

    Statistical analysis

    Patients’ descriptive characteristics of the 2004 and 2008 sample were calculated (SPSS V.18.0). After weighting for the sample frame, the total study sample—that is, both discharged and deceased patients—was representative of the total Dutch population of hospitalised patients.

    A multilevel logistic regression analysis was performed (MLwin 2.22) in order to analyse changes in the rates of patient admissions with at least one AE or preventable AE between 2004 and 2008, while at the same time correcting for clustering on the hospital and hospital department levels.23 The second-order penalized quasi-likelihood (PQL) estimation method was used. To make results comparable with studies that use rates of AEs and preventable AEs per thousand admission-days, we also analysed rates per thousand patient-days with a multilevel Poisson regression analysis. In the multilevel analyses, the eight overlapping hospitals were allowed to covary. Adjustments were made for the stratified sample to correct for over-representation of deceased patients and hospital types. To account for the possibility that changes over time were influenced by changes in the patient mix; terms were added to the model for age, sex, urgency of admission, admission to a surgical unit and main ICD9 diagnostic groups. All variables included in the model were centred to reference values for all Dutch hospital admissions in 2008. In previously published AE rates, we chose to use less complex analytic models which were easier to interpret and suitable for the descriptive analyses of the baseline study. More complex models were required to compare the 2 years in this article; by comparison, the proportions of the more complex models are systematically lower.

    To analyse the changes in group variance for hospital and department levels, we calculated intraclass correlation coefficients (ICCs), the ratio of the between-group variance and the total variance. A higher ICC at the hospital level, for instance, means a smaller variance for all AE rates within the hospitals and a larger variance between hospitals. ORs for the covariables were calculated to ascertain the risk of experiencing an AE for specific groups.

    The characteristics of the AEs, such as type of AE (surgery, drug/fluid, medical procedure, diagnostic, other clinical management, discharge or other) and degree of disability were analysed in SPSS by pooling the 2004 and 2008 data.

    Results

    Patient records and sample

    In total, 11 949 of the 12 400 sampled records were reviewed, 7926 patient records in 21 hospitals from 2004, and 4023 patient records in 20 hospitals from 2008. The 451 (3.6%) records that were not reviewed were unavailable or inadequate for review. During analysis, 66 patient records were excluded because they could not be linked to the hospital administration database, and consequently, had missing ICD9 main diagnostic group information. In total, 11 883 patient records were analysed, 7887 of 2004, and 3996 of 2008.

    Patient characteristics of both samples and total inpatient hospital populations are given in table 1. The total inpatient hospital population stayed relatively stable, with a slight decrease from 1 343 234 patient admissions in 2004 to 1 332 602 in 2008, hospital deaths decreased from 42.329 in 2004 to 35.721 in 2008. The length of hospital stay decreased between 2004 and 2008 in both the sample and the total inpatient hospital population. The percentage of urgent admissions in the 2004 and 2008 sample was comparable. The mean age had increased slightly.

    View this table:
    Table 1

    Comparison of hospital and patient characteristics of the study samples 2004 and 2008, and the total Dutch inpatient hospital population (source: Prismant) in 2004 and 2008

    Rate of patients experiencing an AE

    The physicians identified one or more AEs in 663 of the 7887 reviewed patient admissions for 2004, and in 467 of the 3996 reviewed patient admissions for 2008, 283 and 198 AEs, respectively, were found to be preventable. These numbers are crude unweighted numbers.

    Multilevel analysis showed that between 2004 and 2008 the rate of patients that experienced an AE, corrected for sampling design and case mix, showed a statistically significant increase from 4.1% (95% CI 3.3% to 5.1%) in 2004 to 6.2% (95% CI 5.0% to 7.6%) in 2008 in the total sample, and also in the subsample of tertiary teaching hospitals and general hospitals (table 2). The incidence of preventable AEs remained relatively stable over time in all groups, with 1.8% (95% CI 1.3% to 2.4%) in 2004 and 1.6% (95% CI 1.2% to 2.3%) in 2008.

    View this table:
    Table 2

    Rate of (A) patients who experienced at least one adverse event (AE) or preventable AE. (B) AEs per 1000 patient-days for the total sample and per hospital type (university, tertiary teaching and general)

    Rate of AEs per 1000 patient days

    A number of patients experienced more than one AE, in total 744 AEs in 663 patients were found in 2004, and 512 in 467 patients in 2008. In 2004, on average, 6.0 (95% CI 4.8 to 7.5) AEs per 1000 patient-days were found; in 2008, 10.5 (95% CI 8.7 to 12.7). In 2004, 2.4 (95% CI 1.8 to 3.3) preventable AEs per 1000 patient-days were found in 2004, and 2.7 (95% CI 2.0 to 3.7) in 2008. The results per 1000 patient-days show the same general view as the rates of patients experiencing (at least) one AE or preventable AE, but with stronger contrasts (table 2).

    Clinical process of AEs

    Of all 1256 AEs found in 2004 and 2008 together, 37.5% were found to be preventable, 4.8% led to permanent disability and 8.6% to death. Of all AEs, 51.7% were related to surgery. AEs related to the diagnostic process and discharge were most often found to be preventable, with the diagnostic AEs also most often leading to death (table 3). Box 1 shows case descriptions for AEs and also illustrates the clinical process.

    View this table:
    Table 3

    Adverse events (AEs) (2004 and 2008 data pooled) by clinical process and proportions judged preventable, leading to permanent disability (excluding death) and contributed to death

    Box 1

    Case descriptions for adverse events (AEs) and preventable AEs

    Non preventable AEs

    • Paralysis of vocal cords and respiratory problems after total hip replacement (surgery)

    • Allergic reaction to medication, not previously known (drug/fluid)

    • Neutropenic sepsis after chemotherapy in correct dosage (drug/fluid)

    • Pneumonia after thoracotomy, resulting in artificial ventilation and antibiotics (medical procedure)

    • Pressure sore on heel, despite adequate preventive measures (other clinical management)

    Preventable AEs

    • Technical inadequate hip prosthesis, resulting in two repositions and reoperation (surgery)

    • Patient deceased due to congestive hart failure, as a consequence of intravenous fluid overload not timely diagnosed and treated after operation (surgery)

    • Inadequate antibiotic treatment for urinary tract infection, resulting in death (not corrected after results of culture) (drug/fluid)

    • Perforation of rectum after enema, leading to abscess, gastrointestinal bleeding and death (medical procedure)

    • Missed diagnosis pulmonary embolism in a 50-year-old female on oral contraceptives after 1 week bed rest for “flu” (diagnostic)

    • Missed diagnosis of incarcerated femoral hernia, died of intractable septic complications from peritonitis (diagnostic)

    • Severe hypoxia due to wrongly connected oxygen tube (other clinical management)

    Variation at the hospital and hospital department levels

    ICC estimates for the hospital and department levels together showed a decrease from 0.138 in 2004 to 0.082 in 2008. This indicated less variation at both levels for experiencing an AE in 2008 than in 2004. In other words, hospitals and departments became more alike in the risk of experiencing an AE.

    For preventable AEs, the ICC at the hospital level and the department level together was comparable in both years, with an ICC of 0.152 in 2004 and 0.149 in 2008. The relative contribution of the department level, however, increased from 0.025 in 2004 to 0.067 in 2008. This indicates that for preventable AEs in 2008 there were more differences between departments and less between hospitals in comparison with 2004.

    Covariables

    ORs for the different groups in our model for AEs and preventable AEs in 2004 and 2008 are shown in table 4. Both years show an increase of AEs and preventable AEs by age. Patients who were urgently admitted had a lower risk of experiencing AEs and preventable AEs. Patients admitted to a surgical unit experienced a higher risk in both years. This risk increased over the years from an OR of 1.54 (95% CI 1.10 to 2.16) in 2004 to 3.32 (95% CI 2.17 to 5.07) in 2008 for experiencing a preventable AE. Patients who eventually died in the hospital were at a higher risk of experiencing an AE or preventable AE in both years. Various main diagnostic groups showed higher ORs in 2004 and 2008 using ‘circulatory system’ category as a reference. Of these, the digestive system, injury and poisoning, genitourinary and neoplasms were the most pronounced (table 4).

    View this table:
    Table 4

    ORs for the covariates of a multilevel multivariable model for adverse event (AEs) and preventable AEs in 2004 and 2008 adjusting for clustering at the hospital level and hospital department level

    Reliability

    There was substantial agreement by nurses for positive screening criteria in both years, with an agreement of 82% and a κ statistic of 0.62 (95% CI 0.54 to 0.69) in 2004, and an agreement of 85% and a κ statistic of 0.65 (95% CI 0.55 to 0.75) in 2008. The agreement between physicians assessment for the presence of an AE was fair in 2004 with an agreement of 76% and a κ of 0.25 (95% CI 0.05 to 0.45), and moderate in 2008 with an agreement of 83% and a κ of 0.47 (95% CI 0.33 to 0.61). Moderate agreement was also found in both years for the determination of a preventable AE (2004, 70% agreement, κ 0.40; 2008, 74% agreement, κ 0.49).

    Discussion

    We found an increase in AE rates, and relatively stable preventable AE rates in Dutch hospitals between 2004 and 2008. Due to a shorter length of stay in 2008, rates of AEs per 1000 patient-days showed an even stronger increase than the rate of patients experiencing at least one AE during an admission. Although more harm related to healthcare occurred, the amount of AEs related to substandard care remained stable. More than 50% of all AEs were related to surgical procedures. The odds of experiencing an AE, or preventable AE, were clearly higher for patients admitted to a surgical unit in 2008 than in 2004. It is not quite clear where this relative shift in AEs originates from. Performing surgical procedures on increasingly older and more complex patients may have been of influence. Our 2004 data, however, do not contain information on the American Society of Anesthesiologists (ASA) status, which prevents us from looking into this all too closely. Patients admitted to a surgical unit, or experiencing a surgical AE, show the same increase in age between 2004 and 2008, as does the total patient group (data not shown). Since 2008, attention for patient safety in surgery has increased a great deal. This is not in the least through attention for surgical checklists, which show promising results and may sort an effect in the years after our 2008 measurement.24–26 Despite the lack of a clear explanation, our results do indicate that there is still apparent room for improvement in the surgical process.

    Further results of this study show that urgently admitted patients in both years had a lower risk of experiencing AEs and preventable AEs; receiving unplanned treatment apparently does not lead to a higher risk of substandard care. Also, differences between departments were larger for the risk of preventable AEs in 2008 than in 2004. This shows that some departments may have improved patient safety while others were lagging behind.

    Our study has several limitations. First of all, reviewing physicians depend on information written in the patient record, which is sometimes limited, and information revealing an AE and its preventability may be lacking. The amount and type of information in the patient record may differ between 2004 and 2008 due to the increasing use of electronic patient records (EPRs) or changes in safety culture, however, reviewers did not comment on greater access to relevant information. In 2008, only one hospital in our sample had almost complete EPRs, all other hospitals were in a transition phase between paper records, scanned paper records and full EPRs, which also differed between departments within hospitals. EPRs, intuitively, are more to the point and staccato, but then may also miss crucial information. Previous research has found that hospitals with greater automation of a hospital's information system may be related with reductions in mortality, complications and costs.27

    Another limitation of this study is its reliability. Our study shows moderate inter-rater agreement for the determination of AEs, as is also common in previous research.5 ,6 ,8 ,28 Also, the number of physicians that reviewed one patient record differed between the years. As mentioned earlier, results of the 2004 study showed that two physician reviewers and a consensus procedure does not lead to a higher inter-rater reliability.17 Moreover, results of the 2004 study showed that more AEs were found after the consensus procedure than with two independent reviews by two physicians, indicating that physicians were more reluctant in their judgment of AEs without support of collegial review. The total effect of using one physician per patient record instead of two (or three) and a consensus procedure, however, is not clear.

    Measuring AEs over time on a larger, national, scale is not yet widespread. Landrigan and colleagues also found that despite efforts a significant and measurable improvement in preventable AE rates over the years is still lacking.13 In comparison, we found remarkably lower AE rates than they did with the GTT, however, by using an extensive decision framework our method is more specific and strict in only scoring AEs that are caused by the healthcare. Benning and colleagues measured AE rates as part of a larger evaluation study of the Safer Patient Initiative and also found no significant improvement in preventable AE rates.29 The absence of improvements in preventable AE rates in these and our own study raises the question as to what extent it is possible to improve preventable AE rates.

    Many factors influence the results of large patient outcome studies, such as the quality of care, changes in patient case mix, data quality and chance.30 A large campaign took place in The Netherlands between 2004 and 2008 introducing approaches from other sectors and improving transparency by developing a national set of performance indicators for hospital care.31 In 24 hospitals, improvements in hospital logistics and patient safety took place. However, only three of these hospitals were present in our samples in 2004 and 2008, so a direct result of this campaign is not expected in our sample. Since 2008, a large national safety campaign started that is specifically aimed at improving patient safety and is directed to all Dutch hospitals. A safety management system with improvements on 10 specific medical themes is being implemented in the hospitals. Given that our study shows higher risks in the surgical process, and that recent studies have shown positive results using surgical checklists, current campaigns should include this type of intervention.24 ,25

    Our analysis is corrected for changes in case mix, although residual confounding by patient complexity cannot be ruled out. For instance, indications for treatment and surgery are extended to increasingly older and complex patients. Also, less complex patients are increasingly treated in day care, which is not present in our sample, thus leaving more complex patients to be admitted to hospital for treatment. Day treatments increased by 8.4% between 2004 and 2008.32 This could be a reason for the increase in AEs in 2008. A global examination of the 2004 and 2008 AEs, however, still shows the same classic pattern of AEs. Still present are preventable AEs related to hospital-acquired infections, haematologic and coagulation problems and chemotherapy.

    In conclusion, our study shows that patient harm related to healthcare is a persistent problem that is hard to influence. Due to the above mentioned limitations, comparison between the 2 years must be made with caution. Improving patient outcomes, such as a decrease in preventable AEs, is the goal of most national patient safety programmes. Measuring AEs during more time periods gives information on the current patient safety situation in hospitals, and stresses the continuing urgency. It also raises questions on the feasibility of a decrease of the preventable AE rates. Our results show that high risks exist in the surgical process; further research is needed to show why these risks increased over time. In other studies, surgical checklists have shown promising results to reduce risk to patients. For the future, evaluation of improvement strategies is also important. Further examination of changes to the type of AEs and preventable AEs and the clinical process could reap benefits.

    Acknowledgments

    We would like to thank everyone who contributed to the study, especially the nurses and physicians who reviewed the patient records, and the 33 participating hospitals and their staff who facilitated the patient records.

    References

    1. ↵
      1. Brennan TA,
      2. Leape LL,
      3. Laird NM,
      4. et al
      . Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324:370–6.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Zegers M,
      2. de Bruijne MC,
      3. Wagner C,
      4. et al
      . Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care 2009;18:297–302.
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. Mendes W,
      2. Martins M,
      3. Rozenfeld S,
      4. et al
      . The assessment of adverse events in hospitals in Brazil. Int J Qual Health Care 2009;21:279–84.
      OpenUrlAbstract/FREE Full Text
    4. ↵
      1. de Vries EN,
      2. Ramrattan MA,
      3. Smorenburg SM,
      4. et al
      . The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008;17:216–23.
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Baker GR,
      2. Norton PG,
      3. Flintoft V,
      4. et al
      . The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ 2004;170:1678–86.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Davis P,
      2. Lay-Yee R,
      3. Briant R,
      4. et al
      . Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J 2002;115:U271.
      OpenUrlPubMed
    7. ↵
      1. Schioler T,
      2. Lipczak H,
      3. Pedersen BL,
      4. et al
      . Incidence of adverse events in hospitals. A retrospective study of medical records. Ugeskr Laeger 2001;163:5370–8.
      OpenUrlPubMed
    8. ↵
      1. Vincent C,
      2. Neale G,
      3. Woloshynowych M
      . Adverse events in British hospitals: preliminary retrospective record review. BMJ 2001;322:517–9.
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Soop M,
      2. Fryksmark U,
      3. Koster M,
      4. et al
      . The incidence of adverse events in Swedish hospitals: a retrospective medical record review study. Int J Qual Health Care 2009;21:285–91.
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Leape LL,
      2. Berwick DM
      . Five years after To Err Is Human: what have we learned? JAMA 2005;293:2384–90.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Vincent C,
      2. Aylin P,
      3. Franklin BD,
      4. et al
      . Is health care getting safer? BMJ 2008;337:a2426.
      OpenUrlFREE Full Text
    12. ↵
      1. Schouten LM,
      2. Hulscher ME,
      3. van Everdingen JJ,
      4. et al
      . Evidence for the impact of quality improvement collaboratives: systematic review. BMJ 2008;336:1491–4.
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Landrigan CP,
      2. Parry GJ,
      3. Bones CB,
      4. et al
      . Temporal trends in rates of patient harm resulting from medical care. N Engl J Med 2010;363:2124–34.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Griffin FA,
      2. Resar RK
      . IHI Global Trigger Tool for Measuring Adverse Events. IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement, 2007.
    15. ↵
      1. Classen DC,
      2. Resar R,
      3. Griffin F,
      4. et al
      . ‘Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff (Millwood) 2011;30:581–9.
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Zegers M,
      2. de Bruijne MC,
      3. Wagner C,
      4. et al
      . Design of a retrospective patient record study on the occurrence of adverse events among patients in Dutch hospitals. BMC Health Serv Res 2007;7:27.
      OpenUrlCrossRefPubMed
    17. ↵
      1. Zegers M,
      2. de Bruijne MC,
      3. Wagner C,
      4. et al
      . The inter-rater agreement of retrospective assessments of adverse events does not improve with two reviewers per patient record. J Clin Epidemiol 2010;63:94–102.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Zwaan L,
      2. de Bruijne MC,
      3. Wagner C,
      4. et al
      . Patient record review of the incidence, consequences, and causes of diagnostic adverse events. Arch Intern Med 2010;170:1015–21.
      OpenUrlCrossRefPubMed
    19. ↵
      1. Smits M,
      2. Zegers M,
      3. Groenewegen PP,
      4. et al
      . Exploring the causes of adverse events in hospitals and potential prevention strategies. Qual Saf Health Care 2010;19:e5.
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Hoonhout LH,
      2. de Bruijne MC,
      3. Wagner C,
      4. et al
      . Nature, occurrence and consequences of medication-related adverse events during hospitalization: a retrospective chart review in the Netherlands. Drug Saf 2010;33:853–64.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Hoonhout LH,
      2. de Bruijne MC,
      3. Wagner C,
      4. et al
      . Direct medical costs of adverse events in Dutch hospitals. BMC Health Serv Res 2009;9:27.
      OpenUrlCrossRefPubMed
    22. ↵
      1. Wilson RM,
      2. Michel P,
      3. Olsen S,
      4. et al
      . Patient safety in developing countries: retrospective estimation of scale and nature of harm to patients in hospital. BMJ 2012;344:e832.
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Twisk JWR
      . Applied Multilevel Analysis: a practical guide. Cambridge: Cambridge University Press, 2006.
    24. ↵
      1. Haynes AB,
      2. Weiser TG,
      3. Berry WR,
      4. et al
      . A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491–9.
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. de Vries EN,
      2. Prins HA,
      3. Crolla RM,
      4. et al
      . Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med 2010;363:1928–37.
      OpenUrlCrossRefPubMedWeb of Science
    26. ↵
      World Alliance for Patient Safety. WHO guidelines for safe surgery. Geneva: World Health Organization, 2008.
    27. ↵
      1. Amarasingham R,
      2. Plantinga L,
      3. Diener-West M,
      4. et al
      . Clinical information technologies and inpatient outcomes: a multiple hospital study. Arch Intern Med 2009;169:108–14.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Thomas EJ,
      2. Lipsitz SR,
      3. Studdert DM,
      4. et al
      . The reliability of medical record review for estimating adverse event rates. Ann Intern Med 2002;136:812–16.
      OpenUrlPubMedWeb of Science
    29. ↵
      1. Benning A,
      2. Dixon-Woods M,
      3. Nwulu U,
      4. et al
      . Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ 2011;342:d199.
      OpenUrlAbstract/FREE Full Text
    30. ↵
      1. Lilford RJ,
      2. Chilton PJ,
      3. Hemming K,
      4. et al
      . Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ 2010;341:c4413.
      OpenUrlFREE Full Text
    31. ↵
      1. Duckers M,
      2. Makai P,
      3. Vos L,
      4. et al
      . Longitudinal analysis on the development of hospital quality management systems in the Netherlands. Int J Qual Health Care 2009;21:330–40.
      OpenUrlAbstract/FREE Full Text
    32. ↵
      Dutch Hospital Data. Kengetallen Nederlandse Ziekenhuizen 2009. 2010. Utrecht.

    Footnotes

    • Contributions Study concept and design: MCdB, ML, CW. Acquisition of data: HA, RJB, MB, ML, FvR, HS, LvdS. Analysis and interpretation of data: HA, RJB, MCdB, ML, PS. Drafting of the manuscript: RJB, MCdB, ML. Critical revision of the manuscript for important intellectual content: HA, RJB, MB, MCdB, ML, FvR, KMS, PS, LvdS, CW. Statistical analysis: RJB, ML, PS. Obtained funding: MCdB, CW. Administrative, technical, or material support: HA, MB, FvR, KMS, LvdS. Study supervision: MCdB, CW.

    • Funding Financial support for the study was received from the Dutch Ministry of Health, Welfare and Sport. The funder had no role in the design and conduct of the study; collection, management, analysis or interpretation of the data; preparation, review or approval of the manuscript.

    • Competing interests None.

    • Ethical approval The study protocol was reviewed and approved by the scientific committee of the EMGO+ Institute, and by the ethical review board of the VU University Medical Center.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    Linked Articles