Clinical safety is an essential factor in health care quality, considering the complexity of clinical practice as well as its organisation. Safe clinical practice requires the achievement of three main objectives: to identify the safest and most efficient diagnostic and therapeutic clinical procedures, to guarantee their implementation for those requiring them and to conduct them correctly and without errors.1 The measurement of the risk associated with hospital care is a key issue for the health system. It influences, among others, sanitary, economic, legal, social and even mass media issues. The concept of risk within the healthcare and the public health fields has some special features. It has been traditionally linked to the study of the causal association2 and to the probability of events happening related to health or its loss, such as death, illness, deterioration, accident, recovery, improvement, etc.3

The adverse event (AE) rate in hospitals has been estimated at between 3% and 17%. Of these, 50% could be considered as preventable.4 These studies were conducted in the USA,5^–7 Australia,8 Great Britain,9 Denmark,10 New Zealand11 and Canada.12 13

The reference study was undertaken in 1984 in New York and is known as the Harvard Medical Practice study.5 This study estimated a 3.7% AE incidence rate studying 30 121 medical histories. The AEs resulted in minor or transitional disability in 70% of cases, in permanent disability in 3% of cases and death in 14% of cases. The aim of the review was not to assess the potential prevention of the AEs but to state the extent of negligence in the AE occurrence.

The adverse event rate contrasts with that reported in other studies using similar methodology, although motivations in those studies were different: to develop national policies for the improvement of health care safety, identifying mistakes, and their seriousness and importance. Thus, in the Quality Australian Health-Care Study,8 conducted in 28 hospitals in New South Wales and South Australia, 16.6% of AEs were reported, 51% of which were preventable.

The differences between the rates reported in the New York and Australian studies14 15 might be due to (1) the different definition of AEs in both studies (threshold for causation definition); (2) the different reasons for the studies and consequently the different types of AE investigated; (3) the fact that both studies were based on the review of the medical records (retrospective studies) but were conducted in quite different periods of study (changes in clinical practice).

The Vincent et al9 study conducted in two London hospitals reported a 10.8% AE incidence rate in 1014 patients admitted to hospital in 1998. Of these, 48% of the AEs were preventable. In the Danish study,10 which was conducted in 17 hospitals and included 1097 patients, the incidence rate was 9%.

The Davis et al11 study conducted in New Zealand in 1995 and the Baker et al12 study conducted in Canada in 2000 reported 12.9% and 7.5% AE incidence rates respectively.

The Michel et al16 study conducted in France with a sample of 8754 patients in 71 hospitals, although a prospective study, reported an AE incidence rate of 6.6 for every 1000 patient-days. The operative definition included only severe AEs.17

The choice of the most suitable epidemiological method to study the AEs is not a trivial question. Several studies 18 19 analysed this issue and they concluded that the choice should be based on the objectives of the study. The aim is to combine the minimisation of the bias and correct identification of AEs with the reproducibility of the value judgment concerning the iatrogenetic nature and/or its preventability.

As our objective was to assess the situation in Spain, we decided to conduct a retrospective cohort study by referring to the analysis of the total hospitalisation of the patients discharged in a period of a week, based on a representative sample of the patients admitted to hospital in Spain, taking into account the size of the hospitals, and in order to estimate the AE incidence, their impact and their preventability.

The objectives of this study were to assess the AE incidence rate and the rate of patients suffering AEs in Spanish hospitals, to determine the AE rate in the pre-hospitalisation period and to describe the immediate determinants of the AEs.

MATERIAL AND METHODS

Design

The study was a retrospective cohort study.

Sampling

Two-stage sampling was carried out, stratified by hospital size and random selection of hospitals until the appropriate sample size was reached, and counting the hospital discharges between the 7 and 13 of May 2005. The sample consisted of all discharges in a week from each selected hospital, being the number of discharges in each stratum proportional to the number of discharges in base population that week. Sample size was estimated in 6500 discharges with a precision of 1.32 and a design effect of 2.

Settings

The sample contained 24 hospitals: 6 of them were small hospitals (fewer than 200 beds) with 451 discharges, 13 were medium-sized hospitals (between 200 and 499 beds) with 2885 discharges and 5 were large hospitals (500 or more beds) with 2288 discharges. The real sample size was 5908 patients discharged between 4 and 10 June 2005 (inclusive), but only 5624 medical records were available for screening.

Inclusion criteria

Patients of any age who had stayed more than 24 h in the selected hospitals under the care of any speciality, who had a clinical record in these centres and who had been discharged between 4 and 10 June 2005 (inclusive).

Monitoring period

The complete index hospitalisation of the patients included in the study (discharged between 4 and 10 June 2005).

Case definition

In view of the non-existence of a universally accepted taxonomy of AEs, an accident was defined as any event causing an injury that can result in a longer hospital stay, disability at the moment of discharge, death or any combination of these. The incident does not necessarily cause either injury or harm but can make them more likely. An AE was defined as any accident related to health care and with a causation score of at least 4. To assess the causation (relationship between AEs and medical care), a scale between 1 and 6 was used that stated the accuracy of this relationship in the reviewer’s opinion. The same method was used to assess the preventability.

Variables studied

1. Variables linked to health care: hospitalisation service, type of admission, stay in days and extrinsic risk factors (urinary catheter, peripheral venous catheter, peripherally inserted central venous catheter, central venous catheter, parenteral nutrition, enteral nutrition, nasogastric tube, oesophagogastric percutaneous catheter, tracheostomy, mechanic ventilation or immunosuppression therapy).

2. Variables linked to the illness or to a procedure: main diagnosis (literal or Code ICD-9CM, International Classification of Diseases, 9th revision clinical modification) surgical procedure (literal or code ICD-9CM), ASA risk.20

3. Variables linked to the patient: age, gender and pre-existing intrinsic risk factors (coma, renal failure, diabetes, neoplasia, chronic obstructive pulmonary disease, immunodeficiency, neutropenia, liver cirrhosis, drug addiction, obesity, malnutrition, pressure ulcer, malformations, heart failure, coronary heart disease or hypertension).

4. Variables linked to impact: hospital stay caused by the adverse event, additional procedures and treatments as a consequence of the AEs, disability or death.

Implementation

The screening guide of the IDEA project (identification of adverse events, a questionnaire elaborated under consensus techniques based on a previous investigation, from a list of conditions similar to the one used in the New York, Utah and Colorado studies)5 6 was used to identify potential AEs. Clinical records that fulfilled at least one of the 19 criteria in the screening guide were reviewed in further detail to characterise precisely the AE using the Modular Revision Form (MRF2).

Review process

At the first stage, the nursing staff or physicians from each hospital examined all selected clinical records seeking AE alert conditions using the screening guide. Afterwards, teams composed of two trained doctors, one medical and one surgical, visited the centres in order to confirm the AE through a detailed review of the clinical report referring to this episode (external evaluation) using the MRF221 22 form (Spanish version). The uncertain cases were re-analysed by the executive committee.

Reviewers training

One or two nurse/s or physician/s from each centre were trained during an 8 h workshop to fill in the screening guide. Ten expert physicians were trained for 3 days to fill in the MRF2 and to work with the IDEA project database. Prior to the fieldwork a concordance study was conducted among the reviewers. The aim of this study was to assess the quality of training during the review procedure, AE identification and characterisation, and to find potential mistakes, defects in their description and differences of opinion. Five evaluators analysed 48 clinical reports selected from internal medicine and another five evaluators analysed 22 clinical reports from surgery, obtaining an agreement level from moderate to satisfactory (κ between reviewers and gold standard from 0.652 to 0.868 in medicine and from 0.431 to 0.784 in surgery). The principal researcher with the study leadership team support agreed the gold standard and supervised the training period of the reviewers directly. In the next 20 days, before the fieldwork, they could ask any question raised as a result of the workshop by phone or e-mail.

Data analysis

We calculated the incidence of AEs and the number of patients with AEs that had been caused and detected during the hospitalisation period studied, and included the incidence of AEs previous to the hospitalisation period studied. We calculated the density of the occurrence of the AEs caused and detected during the hospitalisation. We also calculated the rate of patients re-admitted because of an AE and the AE rate occurring during the pre-hospitalisation period for all the patients (Primary Health Care, External Consultation or previous admission to hospital). Finally, we calculated the percentage of preventable AEs.

Univariate analysis was conducted to describe the sample (average, median, standard deviation and interquartile amplitude for continuous variables and frequencies for categorical variables); bivariate analysis was used to state the relationships among the variables (using the Mann–Whitney U test to compare averages, and the χ²-test to compare ratios) and a stepwise logistic regression model using the likelihood ratio test to control the confusion and/or the interaction among them. The hypothesis contrasts were bilateral with a 0.05 significance level, except for the logistic regression model in which we used a p value below 0.05 for inclusion and below 0.10 for exclusion. The statistical analysis was conducted using the statistical programme SPSS version 12.0.

Confidentiality and ethical aspects

This study was conducted following the recommendations of the WHO (World Health Organization) and the Cohesion Law of the SNS (Spanish National Health System).23 The necessary conditions to guarantee the enforcement of the Organic Law 15/1999 of Personal Data Protection were established. The initial data collection was nominal but the individual identification was exclusively maintained until the quality controls of the database were passed. From this point on, a database under the exclusive control of the director of the study allowed a link between the data and the patients. All the participants in this study were obliged to maintain the confidentiality of the information they had access to during the study, as well as in any other professional activity. The data were presented in such a way that patients could not be identified from the results.

The study was presented to the Clinical Investigation and Ethics Committee of Aragon for approval.

RESULTS

The estimated sample size was 6500 patients from 24 hospitals. The real number of discharges during the week under study was 5908 patients. During screening, hospital staff found that 103 clinical records were missing. When the external reviewers completed the MRF2, they found that the clinical information from 181 cases could not be retrieved and therefore the total number of patients studied was 5624. Among those 5624 patients, 1755 (32%) were screened as potentially having AEs and 3869 were ruled out because they did not show any of the screening guide alerts. On reviewing the patients with positive screening, the experts found 501 false positives and only 191 patient records showed incidents.

Thus, 1063 patients with AEs related to health care or to illness development were detected (fig 1). The incidence of patients with AEs related to health care was 9.3% (525/5624, 95% CI 8.6% to 10.1%). The incidence of patients with AEs directly related to hospital care (excluding those from primary care, specialised outpatient consultations and those caused in another hospital) was 8.4% (473/5624, 95% CI 7.7% to 9.1%).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Adverse event (AE) distribution and their relation to the health care. *Reviewers could not complete the screening guide. †Reviewers could not complete the MRF2 form.

Among the patients with AEs, 17.7% suffered more than one AE. In 105 (22.2%) of the 473 patients with AE related to hospitalisation, the AE required hospital readmission.

A total of 13.2% of patients with intrinsic risk factors suffered AEs compared with 5.2% of patients without these risk factors. The difference reached statistical significance (p<0.001). A dose–response effect was detected: 10.5% of patients with an intrinsic risk factor suffered AEs, this rate rose to 15.1% in patients with two risk factors and to 22.9% in patients with three or more risk factors. The trend reached statistical significance (p<0.001).

In the same way, 9.5% of patients with an extrinsic risk factor suffered AEs against 3.4% of patients without these risk factors. The difference reached statistical significance (p<0.001). Since a high percentage of patients had a peripheral catheter in place the analysis was repeated without considering this circumstance as a risk and the effect was the same. A dose–response effect was also detected so that 5.6% of patients without extrinsic risk factors suffered AEs; this figure rose to 11.4% in patients with one risk factor, to 14.2% in patients with two risk factors and to 33.5% in patients with three or more risk factors. The trend reached statistical significance (p<0.001).

The mean age of the patients who developed AEs during hospitalisation was 64.3 years (SD 20.5) with a median age of 71, against a mean age of 52.5 years (SD 25.0) with a median age of 57 for patients without AEs (p<0.001). No gender differences were found.

Patients over 65 suffered AEs more frequently than those patients under 65 (12.4% vs 5.4%). The risk of suffering AEs for patients over 65 was thus twice as high as the risk for patients under 65 (RR 2.5 95% CI 2.0 to 3.0). The difference reached statistical significance (p<0.001).

Multivariate analysis showed that the age, the length of hospitalisation, the hospital size and the number of risk factors both intrinsic and extrinsic, explained the occurrence of AEs (table 1).

The risk of suffering AEs for patients admitted to small hospitals was 1.4 times higher than the risk for patients admitted to large hospitals. The risk for patients with intrinsic risk factors was 1.6 times higher than for patients without them. Patients with no extrinsic risk factors and over the age of 65 had twice the risk of those under 65. The risk for patients under 65 with extrinsic risk factors was 2.3 times higher than the risk for those who did not. Among patients over 65, the effect was less so that patients having extrinsic risk factors had a 1.3 times higher risk than patients who did not (effect of the interaction between both variables). The risk for patients who were admitted for longer than a week in a surgical service was 5 times higher than the risk for those who were admitted for less than a week, and also the risk for those admitted for longer than a week in a medical service was 2.8 times higher than the risk for those whose length of stay was shorter (effect of the interaction).

According to hospital size, significant statistical differences were detected in the patients with hospital care-related AE distribution. The AE incidence was higher in small hospitals (10.2%, 95% CI 7.41% to 13%), intermediate in large hospitals (9.66%, 95% CI 8.45% to 10.9%) and lower in medium-sized hospitals (7.14%, 95% CI 6.20% to 8.08%). The incidence of patients with AEs in medical services was higher than in surgical ones (8.86 vs 8.07).

The incidence of AEs occurring during hospitalisation was 1.2 AEs for every 100 patient-days (95% CI 1.1 to 1.3). The incidence of moderate or severe AEs was 5.6 AEs for every 1000 patient-days (95% CI 4.9 to 6.3). Data of hospital size and service type are shown in table 2.

Among all the AEs, 66.3% required additional procedures (eg, radiodiagnosis test) and 69.9% required additional treatments (eg, medication, rehabilitation or surgery).

With regard to outcome, 31.4% of AEs resulted in a longer hospital stay. The incidence of death in patients with AEs was 4.4% (95% CI 2.8% to 6.5%).

All the AEs were divided into six categories. Of the AEs, 37.4% were related to medication, 25.3% to any type of healthcare-related infections and 25% to technical problems during procedures. Detailed data about the type of AEs observed are shown in table 3.

42.6% of AEs were considered preventable.

In 19% of the cases, the reviewers considered that the information on the AE in the record was inadequate or not very adequate.

DISCUSSION

The ENEAS study is part of the group of studies whose objective is to improve healthcare quality. In order to find the highest number of improvement opportunities, the methodology considers that a patient could suffer several AEs during hospitalisation. We included in this analysis those AEs caused in the pre-hospitalisation period and detected during the hospital stay and also those suffered in a previous admission and which caused readmission.

Most of the studies published until now have only considered the most serious AEs in each patient case. Unlike most studies, our study recorded all the adverse events suffered by the patient, leading to a more precise incidence estimate. This detailed approach allowed us to calculate not only the cumulative incidence of patients with AEs but also the overall incidence of AEs and the AE incidence rate per 100 days of admission, thus enabling us to assess the severity and preventability of each event.

The incidence of patients with AEs related to hospital care (8.4%, 95% CI 7.7 to 9.1) and to health care (9.3%, 95% CI 8.6% to 10.1%) were similar to those obtained in the studies aimed at improving quality. The results did not differ from the adjusted Australian, London, Danish, New Zealand and Canadian studies, but they were higher than the American studies and lower than the non-adjusted Australian study (fig. 2). These results are in agreement with the methodology used.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2 Incidence in the main studies. NY, New York; UC, Utah and Colorado; A, Australia; L, London (Great Britain); D, Denmark; NZ, New Zealand; CA, Canada; Spain.

The average age of the patients with AEs was 12 years higher than those without AEs. This result is consistent with most studies. The risk of suffering an AE for patients over 65 was twice as high as for patients under 65 (RR 2, 95% CI 1.5 to 2.6), this result is consistent with Harvard Medical Practice study.5 In line with the other studies, no gender differences were found.

The association between the seriousness of the patient condition and the incidence of patients with AEs could not be analysed because no variable provided this information explicitly. Nevertheless, we approached this assessment by analysing the association with age, the length of hospital stay and risk factors, both intrinsic and extrinsic, that can be considered as indirect signs of seriousness.

This study showed that the level of patient vulnerability was a decisive factor in the occurrence of AEs linked to health care. It was possible to explore this association because measurements of intrinsic and extrinsic risk factors were made in all patients in the study, whether or not they suffered AEs. Until now, this has only been suggested. Thus, the higher the number of risk factors, the higher the risk of suffering AEs. The result concerning the relation between the occurrence of AEs and the presence of extrinsic risk factors is even more interesting because even if it is difficult to act on intrinsic risk factors, as they are factors which are not easy to modify, it is still possible to act on the extrinsic ones. If it is possible to reduce them to the minimum in each patient, the risk of suffering AEs will be minimised considerably.

The fact that some patients suffered more than one AE (17.7%) suggests that the presence of an AE seems to make the occurrence of more events more likely. Likewise, patient vulnerability may determine an increase of the same risk. The characteristics of our study do not allow us to verify these statements.

The prognosis (main condition resulting in a potential residual disability) and the existence of comorbidities are related to the seriousness of the AEs, although not in a linear way. These results are similar to those from the Michel et al’s study.16

Incidence differences were found depending on the hospital size. The incidence is higher in small hospitals and lower in the medium-sized ones when compared to the incidence in large hospitals. This result is slightly different from the study of Baker et al12 conducted in 20 hospitals in Canada. In that study, the incidence increased with hospital size. The result in our study may be conditioned by the number of patients studied, being proportionally smaller in small hospitals, so the CI of the estimated incidence is very wide and may be influenced by other determinant variables of AEs that have not been considered.

Our results make it possible to establish the relation between length of hospital stay and AE. Thus, on the one hand, it is a risk factor (adjusted by the other variables) because the longer the stay the higher the risk of suffering AEs, and ,on the other hand, it is a consequence of the AE, since its occurrence makes the stay longer.

The higher incidence rate of small hospitals may be conditioned by the shorter average stay, making the denominator proportionally significantly lower. This result could also be explained by possible information bias, by differences when filling in the clinical record and by the characteristics of the patients. In any case, it should be considered with caution because the estimate is not very accurate; this is an aspect to study in detail in the future.

The incidence rate in large hospitals is higher than in medium-sized ones, and is probably related to the higher complexity of clinical practice, although, nowadays, size may not be an appropriate characteristic to classify hospitals. Thus, it might be more interesting to look for the combination of available technology and clinical practice complexity, in order to classify the types of hospitals from the point of view of the clinical safety of patients and the AEs linked to health care.

Only the French study offers data on the incidence rate, but only study adverse events were studied.16 The results, after adjusting the methodology of both studies, are perfectly comparable (French 6.6 per 1000 days, 95% CI 5.8 to 7.41, vs Spanish 5.6 per 1000 days, 95% CI 4.86 to 6.28).

Although the percentage of AEs generated during the pre-hospitalisation period and detected during hospitalisation was lower than the percentage found in the Canadian study by Baker et al12 (13.8% vs 31%) it is not negligible. Furthermore, 20.6% of all the AEs took place before admission (including those caused in primary care and external consultation). This fact is especially significant in the Spanish National Health System, with its high accessibility and well-developed primary care. Healthcare-related infections, surgical procedures and problems linked to the use of medication explain 70.4% of AEs occurring during the pre-hospitalisation period, emphasising the fact that problems related to medication involve 34.8% of AEs.

It would be interesting to study in depth all AEs linked to primary care, not just those resulting in hospitalisation. The same may be said about the AEs associated with the emergency department.24 Although this study was not designed to study this aspect, it brings forward some data that allow the recommendation of this analysis strategy. In total, 9.6% of AEs from the pre-hospitalisation period took place in the emergency department during medical care prior to the day of admission. Several AEs that occurred before admission to the ward happened in the emergency department (37.5%).

Concerning the nature of the problem causing the AE, this study identified possibilities of improvement barely considered before. The not inconsiderable figure of 7.6% of AEs related to ward care is in fourth place after the AEs related to medication, to nosocomial infections and to surgical procedures, and even their incidence might have been underestimated. On the other hand, there are several proved efficacy strategies designed to diminish all of these AEs. The challenge seems to be to put them into practice. The diagnostic error rate might also be underestimated, maybe because there is no specific screening criterion about misdiagnoses. Furthermore, in general, most admissions take place with a provisional diagnosis or with the reason for consultation as diagnostic, so that it is difficult to find a diagnostic error when reviewing the clinical record.

Although this is not a study specifically designed to analyse the adverse events linked to the use of medication, it has been shown to be highly effective for this purpose. This group involves the highest frequency among the different types of AEs. It is therefore possible to state that 4.1% of the studied patients admitted to hospital suffer some AEs related to the use of medication. This result is twice the one found by Bates et al25, although it is lower than the result found by Otero et al.26 This result is especially significant in the case of medical services, as Alcalde et al27 have shown.

The use of medication is a complex system where professional expertise, teamwork precision and individual patient susceptibility interact. That is why process management is an appropriate response for the improvement in quality of care, because it enables us to state the components of the process, their relations and the activities to guarantee success for safety of the patient.28

Dealing with AEs related to the use of medication may be performed either using an individual approach in which it is hoped to establish causal attribution, human mistakes and system failures29 30 or from a collective point of view that makes it possible to identify the risk factors as much as the characteristics associated with the AEs in a group of patients. This study is aligned with the second group.

Among the limitations of the study is that AEs were identified by means of information from clinical records; poor quality record-keeping could have led to underestimating the incidence of AEs. Furthermore, the primary healthcare record was not available, so the assessment of preventability was more difficult. With regard to the quality of the notes in the clinical record, the reviewers considered that the information about the AE was inadequate or not adequate in 19% of the cases.

The Spanish version of the modular reviewing form MRF2 has been adapted by the IDEA project.31 32 It is a questionnaire in which the reviewer is required to make some value judgments and therefore he/she must be an expert on the subject, able to detect AEs by means of implicit criteria in most cases. Furthermore, the specificity of the medical or surgical process may have made the exhaustive characterisation of the AE more difficult. That is why training and a concordance study have been carried out in which higher values than those of American and European studies were found. The questionnaire has been classified as moderately reliable,33 whereas in our study the pre-test reliability is considered moderate to good.

Using stage 1 reviewers from participating hospitals could make the screening guide less sensitive; however, it was positive in 31.2%, greater than that found in US studies, but 10% smaller than the other studies. This could be because we were very restrictive with the first criterion of the screening guide (prior hospitalisation). The external reviewers were clinical experts unconnected with the service studied, and therefore were unaware of the working style, task and service organisation, existence of work protocols or clinical practice guidelines, etc, which, in some cases, has made it difficult to know the final circumstances causing the AEs and therefore the potential for preventability. In contrast, specific training in the evaluation of AEs and the impartiality of the assessment as external professionals have their advantages and reduce the bias of wrong case identification, which means that the internal validity of the study increases.