Article Text
PDF
Abstract
Background Interest in patient safety (PS) is growing exponentially, fuelled by epidemiological research unveiling the extent of unsafe care. However, there is little information about the frequency of harm in developing and transitional countries. To address this issue, the authors performed a study known as the Iberoamerican Adverse Event Study, through a collaborative between the governments of Argentina, Colombia, Costa Rica, Mexico and Peru, the Spanish Ministry of Health, Social Policy and Equality, the Pan American Health Organization and the WHO Patient Safety.
Methods The study used a cross-sectional design, involving 58 hospitals in the five Latin American countries, to measure the point prevalence of patients presenting an adverse event (AE) on the day of observation. All inpatients at the time of the study were included.
Results A total of 11 379 inpatients were surveyed. Of these, 1191 had at least one AE that the reviewer judged to be related to the care received rather than to the underlying conditions. The estimated point prevalence rate was 10.5% (95% CI 9.91 to 11.04), with more than 28% of AE causing disability and another 6% associated with the death of the patient. Almost 60% of AE were considered preventable.
Conclusions The high rate of prevalent AE found suggests that PS may represent an important public-health issue in the participating hospitals. While new studies may be needed to confirm these results, these may already be useful to inspire new PS-improvement policies in those settings.
- Adverse event
- patient safety
- research
Statistics from Altmetric.com
Introduction
In the past few years, interest in patient safety (PS) has grown exponentially, fuelled by epidemiological research unveiling the extent of unsafe care to patients. The impact of some of the first major studies of these issues1 2 summarised in the To Err is Human3 report still reverberate today. Similar studies have been carried out in the UK,4 Denmark,5 Spain6 7 and elsewhere.8–12 The frequency of adverse events (AE) attributed to unsafe hospital care in the countries studied to date has been considerable.13 However, the vast majority of this research comes from developed countries, and there is an obvious need for priority setting in transitional and developing countries, where relatively little information about the frequency of harm has been available, and nearly all of that has come from single-country studies.14
Although the risk of harm has always been present in the clinical environment, quantification of this harm through the first research studies has been instrumental for policy change. Prior assessments of the magnitude of harm played a major role in motivating and informing the development of new policies focused on improving safety.
Research thus represents the first step in bringing about change, by identifying and prioritising problems.15
We performed this study, known as the Iberoamerican Adverse Event Study (IBEAS, in Spanish), with the main objective to determine the prevalence of AE in acute hospitals of five Latin American countries and also their type and relation with some specific factors.
This study is the result of a collaborative between the governments of Argentina, Colombia, Costa Rica, Mexico and Peru, the Spanish Ministry of Health and Social Policy, the Pan American Health Organization (PAHO) and the World Alliance for Patient Safety of the WHO.
Methodology
This is a cross-sectional design study using a purposive sample of 58 secondary and tertiary acute care hospitals of Argentina, Colombia, Costa Rica, Mexico and Peru. Hospitals were of intermediate complexity, implying the availability of surgical theatres and postoperatory observation units. The selection of hospitals was voluntary and based on feasibility, because the intent of the study was to demonstrate the existence of PS concerns in hospital care rather than to draw any national or regional inference. The only additional criterion was to reach at least 2000 patients per country, in order to allow for local policy making, with a minimum level of precision of 1.5% for an estimated prevalence rate of 10% and a loss of 5%. All patients admitted to the participating hospitals at the time of the study (1 week in late 2007) were surveyed, irrespective of the reason for hospitalisation, their condition or area of hospitalisation.
An AE was defined as any event causing harm to the patient that was perceived to be more related to the healthcare management rather than to the patient's underlying condition.6 In this study, clinical consequences in patient describe the AE.
The study estimated the point prevalence of patients showing an AE. A prevalent AE is present on the day of observation (active, in treatment or in after-effects stage) and could have occurred during the hospitalisation or before the hospitalisation.
Data-collection tools
Screening review form (SRF) which looked for the presence of 19 trigger or criteria. The SRF was directly adapted from the Spanish IDEA study16 and similar to those used in other studies.4 It included data about some intrinsic or comorbidity factors and extrinsic risk factors (catheters and other medical devices in patient). The SRF can be completed by a registered nurse or a clinical physician.
Spanish version of the modular review form (MRF217) adapted from the Spanish national study of AE (ENEAS). This form is completed by a physician with at least 5 years' clinical experience. Physicians could be from the same hospital (owing to the long distances between hospitals in some countries) but they abstained from reviewing cases from their own unit. This stage allows an assessment of the attribution and preventability of the AE, its impact on the patient and also identification of opportunities for improvement.
Variables studied included hospital complexity (secondary or tertiary), department, patient's age, intrinsic risk factors or comorbidity, external risk factors related to clinical devices (see SRF as online appendix), type of hospital admission (planned or urgent), length of stay at hospital before the survey, type of AE (related to medication, infection, etc—see A7 in MRF2) and origin of AE (prior to hospitalisation or during hospitalisation, etc). The selection of these variables was according to the findings of the previous AE studies.
The data-collection instruments were adapted to the Latin American context through a working session held with a representation of researchers from the five countries involved, including clinicians, epidemiologists and managers. In this respect, certain risk factors were included, such as prematurity, alcoholism, peritoneal dialysis and haemodialysis. A glossary of terms and abbreviations with additional clarifications was compiled, as well as a manual with all the operational definitions (these materials are available upon request).
An ad hoc data-management application was also made available: AE Surveillance and Control System based on the IDEA Project database;16 SVCEA 1.0—IDEA 4.0 database.
Training of reviewers
The reviewers training took place in two stages. First, the trainer workshop addressed the national coordinating teams. Second, the national coordinators trained in turn the national investigators. A concordance study was carried out using clinical records from each country.
Data-collection method
The methodology was similar to that used in retrospective studies, but patients were screened for AE only in the 24 h immediately prior to the review process, regardless of when the patient was admitted.
The SRF was completed by nurses or physicians for all hospital inpatients on the day of observation. Whenever a patient screened positive, a trained physician filled out immediately part of the MRF2, taking care to complete the causality assessment module (on a scale of 1–6). The reviewers, when they could not find any data in the medical record, could ask the physician in charge of the patient about the missing data. After patient discharge, or 30 days after the first observation (even if the patient was still hospitalised), the physician reviewer completed the MRF2 by assessing the AE preventability (on a 1–6 scale) and impact. An AE due to healthcare was established as any AE with a score greater than 3 on the causality scale. The same threshold was used to define preventability. A severe AE was defined as one involving the death of the patient, causing permanent disability or requiring additional surgery. A moderate AE caused a lengthened hospital stay. All others were considered slight AE.
All hospitals started the data-collection period on the same day and completed the first stage within 1 week. If, on the day of observation, the patient was not available in their room, but the case notes were available, the study proceeded normally. If the case notes were not available, the study was postponed until the end of the day, when the bed would be considered empty if the case notes remained unavailable. All patients were counted and reviewed only once, despite some inpatient transfers between wards, which occurred during the observation period and had to be identified by the reviewers for not studying the same patient twice.
Data analysis
The data were entered into the centralised AE Surveillance and Control System database managed by the principal investigators. After cleaning, the data were exported to SPSS 14 for statistical analysis.
A descriptive analysis explored the variable distributions, through a bivariate analysis (χ2 test or Fisher exact test for the categorical variables and the Student t test or Mann–Whitney U test for numerical variables, as well as analysis of variance). Finally, logistic regression models were constructed to explain the prevalence, and preventability of AE, and a multinomial model was conducted to explain their severity (severe vs slight and moderate vs slight). Models used independent variables related to hospitalisation, characteristics of patients and characteristics of AE.
Confidentiality and ethics issues
The study steering committee ensured that the relevant national standards for the protection of human subjects and personal data were respected. The study maintained data anonymity and confidentiality, and complied with the ethical principles of the Helsinki Declaration and other related bodies. The study was approved by the PAHO Ethics Review Committee and the corresponding national ethics committees of the participating countries.
Results
Pilot study
In order to identify some difficulties and to control possible variations in reviewer's training, a pilot study was conducted in each country. Table 1 shows the kappa index obtained as a measure of the agreement between the reviewers and the national coordinating team in three decisive points: screening, identifying an AE as related to healthcare and preventability of the AE.
Kappa index for screening, identification and preventability of adverse events
Sample characteristics
The study sample included 11 379 individuals registered as inpatients on the day of the study at the 58 participating hospitals. For 47 inpatients, it was not possible to complete the review in a second stage because medical records were unavailable; these patients were excluded from further study.
The median age of the patients surveyed was 45, with an IQR of 39. Table 2 shows the sample composition stratified by country of origin. Countries 1, 3 and 5 had a slightly higher median age. The sex distribution varied somewhat among countries as well, particularly in country 1 (where the proportion of women was lower) and country 4 (where it was higher).
Characteristics of the study population
Clinical wards in hospitals were classified in the following categories: medical specialties, surgery and gynaecology, obstetrics, paediatrics and intensive care. Countries 1 and 2 had the highest number of paediatric patients (the age range of paediatric patients varied per country). Country 4 had fewer intensive care patients.
The type of admission (urgent or planned) was not always available. Country 4 showed the highest number of planned admissions and also the shortest inpatient stay prior to the survey (3 days on average, although these data were collected for about 15% of patients).
Country 5 had the highest frequency of patients with intrinsic risk factors, and Country 2 the lowest. Countries 3 and 5 had the lowest rates of use of medical devices such as catheters that increase the likelihood of micro-organism introduction or complications.
Prevalence of AE
Of the 11 379 patients who were screened for AE, 3853 (33.9% of all patients) fulfilled at least one of the SRF criteria. Of those, 1754 (45.5% of those screened and 15.5% of all patients studied) were considered to have experienced some harm or complication after completing the MRF2. Among these, in 1191 patients the reviewer judged the harm to be more related to the care received, rather than to underlying patient conditions. The prevalence of patients with at least one AE was thus 10.46% (95% CI 9.91 to 11.04), and the AE ratio (the number of AE over the total number of patients) was 11.85% (1349 AE).
The range of AE rates in the five countries varied from 7.7% to 13.1 (SD: 0.058). Prevalence varied also according to hospital with a median of 9.7% and an IQR from 6.5% to 13.2%.
Table 3 illustrates the likely contribution of different variables to the prevalence of patients with AE. More AE were identified in surgical departments than in medical specialties, and this also occurs in obstetrics, paediatrics and intensive care. Patient age was not retained as an independent variable or as a confounding factor in the final model. This suggests that age-associated risk may be a factor of other variables such as the length of stay, the comorbidity, and the type of care received. Although it was difficult to establish categories of hospital complexity (most participating hospitals provided a broad range of services including surgical specialties), it seems there is an association between increasing hospital complexity and higher risk of AE prevalence. Both urgent admission and longer prior stay (treated here as a continuous variable) increased the risk of AE. Likewise, patient comorbidity and greater use of invasive devices are associated with a higher risk of AE.
Correlates of adverse events in multiple logistic regression analyses
Characteristics of AE
Of all AE, 181 (13.4%) were related to patient management and nursing care, 111 (8.2%) medication, 501 (37.1%) to hospital-acquired infections, 385 (28.5%) to surgical procedures and 83 (6.1%) to the diagnosis. The five most frequent clinical consequences of the AE were 127 hospital-acquired pneumonia (9.4%), 111 surgical-wound infections (8.2%), 97 pressure ulcers (7.2%), 86 other complications related to surgery or procedure (6.4%) and 67 sepsis (5%). These altogether accounted for 36.2% of all AE.
Impact of AE
Overall, 64.7% of all AE (780) prolonged the hospital stay by an average of 16.1 days. The prevalence of patients whose complete admission was due to an AE was 1.57–179/11 379 (95% CI 1.34% to 1.80%). The proportion of AE associated with any kind of disability was 28.8%, and with the death of the patient through a causal relation, 5.8%.
Of the AE identified, 259 out of 1309 (19.8%) were considered severe (related to death or causing disability at the discharge or deserving surgery for its correction), whereas 21.5% of the AE (281) did not even prolong the hospital stay.
The variables in the multinomial model that illustrate the impact of AE on patients' health and on resource consumption (prolonged stay and readmissions) are: patient's age, prognosis of the index condition, duration of hospitalisation prior to the AE and whether the patient is in surgery or intensive care. AE that occurred during procedures, before admission or at discharge, and AE categorised as nosocomial infections, complication of procedures and error or delay in diagnosis were also associated with a worse impact (table 4).
Severity of adverse events (AE) assessed with multinomial logistic regression models: two models explaining moderate and severe AE versus slight AE
Preventability of AE
About 59% of all AE (674/1144) were judged preventable. The prevalence of patients having at least one preventable AE was 5.49% (95% CI 5.07 to 5.91). In the two most frequent categories of AE, those related to nosocomial infections or procedures (accounting for two-thirds of all AE detected), 60% and 55%, respectively, could have been prevented.
The variables explaining the preventability of AE (table 5) are: the complexity of the hospital where the patient was admitted, whether admission was planned, the type of hospital specialty (higher preventability in obstetrics and medical specialties) and AE related to clinical procedures or diagnosis. Neither the sex nor the ASA risk nor the prognosis of the index condition explained the variability in the model. Only the patient's age was included in the model with a correlation with AE preventability, being higher in children younger than 1 year. It also seems that preventability rises as the patient ages. The severity of the AE was not associated with its preventability.
Correlates of preventable adverse events assessed using a multiple logistic regression model
Discussion
We evaluated the prevalence of AE in selected hospitals in five Latin American countries, being the largest study in that Region. The results suggest that PS is a significant problem in healthcare in these countries, underscoring what has previously been shown in other health systems in more industrialised economies. At least 10.5% of hospitalised patients experienced an AE during their stay. This figure refers only to AE that were present on the day of observation. The rate of patients who may have suffered any AE during their entire hospitalisation was therefore likely higher. In addition, this is the first international study designed to measure the point prevalence of unsafe care in hospitals. The findings obtained here are therefore not directly equivalent to those obtained in studies analysing the retrospective incidence of AE throughout the course of hospitalisation.
The nature of prevalent AE identified through this study suggests that hospital-acquired infections are among the main problems occurring in the participating hospitals. This type of AE accounts for approximately 40% of the cases. The study design may explain in part this result, given that because of the longer duration of such infections compared with other shorter-lived AE, it would likely have been more easily identified on the day of observation.
The high frequency of AE that causes admission to hospital (1.5 out of 100 inpatients) underlines the importance of studying other levels of care, as well as the interface between them, and the patient follow-up mechanisms to detect potential complications after discharge from any healthcare setting.
Both the presence of comorbidity and the use of medical devices were independently associated with a higher risk of prevalent AE, as were the length of hospitalisation prior to the observation and the unit or department in which the patient was being treated. This observation supports the relationship between healthcare management and the occurrence of an AE. However, some of these risk factors, such as the presence of invasive medical devices or the department in which the patient is treated, may vary with time. These circumstances should be taken into account in future projects in order to properly assess the role played by these variables in the causality, severity and preventability of the AE, rather than deeming them invariable throughout the patients' stay.
It is difficult to establish parallels between the rates obtained in this study and others which use the traditional retrospective record review. Nevertheless, the proportion of severe AE (almost 20%) seemed slightly higher than other prior studies such as the Spanish Estudio Nacional de los Efectos Adversos (ENEAS)7 (16%), as the proportion of moderate AE (increase the stay), whereas the proportion of minor AE (21.5%) was low compared with the ENEAS study (45%), which was a retrospective incidence study in which the same forms and variable definitions were used. Approximately 6% of the AE were directly related to the death of the patient. This figure seems higher than in other studies performed in more developed countries and addressing the entire period of hospitalisation: the Canadian Adverse Events Study (CAES)11 (4.7%) or ENEAS (2.3%).7 This observed difference may suggest that the AE occurring in the selected hospitals were on average of higher severity. However, this could also indicate a different pattern in the nature of AE being identified through a prevalence design. It would thus seem convenient to understand the nature of the AE identified through each major study design, and to conduct comparative studies to correctly evaluate the occurrence of severe (ie, lethal) AE.
The assessment of preventability is a function of the perceived adequacy of the care provided, given the existing standards of care and the resources available. Despite the higher intrinsic vulnerability of both infants and older people, it is interesting to note that the reviewers considered that the AE occurring to patients in these age ranges were preventable at a higher rate, reinforcing the notion of some concerns with the services provided to such populations.
It is also important to note that, as previously mentioned,7 the preventability of an AE is not a function of the severity or duration of the AE (or their prevalence). Owing to the study design, the AE identified were of a longer duration, more severe and equally preventable compared with those detected in an incidence study.
As discussed above, selecting the most appropriate epidemiological design is an important matter. The design must follow the objectives of the study, trying to minimise the risk of biases, ensuring the validity of AE detection and the reliability of the assessments about the causality and preventability of an AE.18–21 The cross-sectional point prevalence design used in this study makes it possible to estimate the magnitude of the problem, despite the higher likelihood of missing minor or short-lived AE, while its running costs (in terms of sampling, retrieving and analysing medical records) seemed to be significantly lower. It also offers the opportunity to complement and double-check some missing data with the attending physicians on site, facilitating the data-collection process, thus making this design less dependent on the quality, archiving and management of the medical records. These presumed advantages may make this design more suitable for low-budget studies as well as for settings with less reliable medical records. Additionally, its lower running costs makes it more attractive for periodic survey and monitoring, allowing panel studies for temporal trend analysis.
Apart from the design of the study, several other circumstances deserve further consideration, since they may influence the results of the study. For example, the differences in the structures of the health systems in various countries (supplies, staffing mix and patterns, infrastructure, etc), as well as the fact that the average age of inpatients in the region seems lower than in other studies, and the use of invasive procedures is not as widespread thus potentially indicating a lower risk of AE. In addition, the hospitals studied were in most instances some of the better hospitals in these countries, so the results may not be representative of all hospitals in the region. The above-mentioned factors, the complexity of care and the proportion of inpatients in each department could also explain the differences found in hospital settings.
There are also differences in the operational definition of AE between this and other studies. (In the IBEAS study, AE occurring at healthcare levels other than the hospital were also counted, as well as those AE that did not prolong the hospital stay.) The quality of the medical records has a definite impact and is one of the primary limitations of this type of research, despite the additional opportunity for clarification by the attending physician encountered in this type of design. The reviewers judged 85% of the medical records as appropriate to identify AE characteristics. An additional limitation is related to the adequacy of the data-collection tools to assess the point prevalence of AE. Although some of the items were less appropriate for the assessment of prevalent events, the percentage of patients screening positive and the checklist's ability to positively detect prevalent AE are fully in line with the findings of the other AE studies. The predictive positive value of SRF was 31% in IBEAS, 20% in CAES11 and 37% in ENEAS.6 We therefore believe that the materials are sufficiently sensitive and are suitable for identifying both prevalent and incident AE.
Regarding reliability of causality and preventability judgements, while the values seem on the lower bound for this validity index, most studies on AE that have estimated it did not reach 0.5: causality—QHACS8 0.42, UTCOS9 0.40, CAES11 0.45; preventability—QHACS8 0.33, UTCOS9 0.24, CAES11 0.69. Although we have presented the kappa results for each country, individual reliability was focused. Training strengthening before the study was required when a reviewer obtained a kappa value below 0.4. The agreement study provided new clues in identifying sources of variation when performing judgements of causality and preventability. Consensus was reached on specific situations and conditions, and inferred if possible. It was established, for example, how to judge a patient's poor compliance or a complication occurring while the patient is on a surgery waiting list.
Conducting a multinational and multicentre study required continuous coordination and oversight. In this study, this was facilitated by a strong country coordination system delivered by PAHO and facilitated by an active web-based Community of Practice and technical supervision through email and periodic face-to-face workshops. These tools proved vital to increase the consensus and reliability across reviewers as assessed in the early phases of the study, reinforcing the idea that conducting such a study requires, in addition to good initial training, adequate guidance and reinforcement of the most difficult aspects.
The IBEAS study was one of the first studies addressing the magnitude of AE in hospital care performed in a number of transitional countries. The design used was novel for this type of study but was shown to be feasible and perhaps advantageous, since it was less resource-intensive than the traditional retrospective record review. The study involved more than 1000 healthcare workers and researchers in the five countries, fostering a network of trained professionals with great potential for further activity in the region in the area of PS. In fact, following the completion of the study IBEAS, significant activities at the national and subnational level, including the issuing of national policy and structures for PS, took place in the participating countries, which in many cases were led by the study team members. It is hoped that the study served to increase awareness about PS and to strengthen the national and regional agendas of work on PS in the participating countries.
Acknowledgments
Thanks to all members of the IBEAS Research Team for their dedicated contribution: JC Fraschina and MF Monteschia, from Argentina; A Franco, N Guillot and DS Sierra, from Colombia; JC Umaña Rojas, M Vargas and EV Mora from Costa Rica; L Anzurez Mosqueda, W Tovar Vera, D Pineda Pérez, R Orozco Zavala, D López Estrada, JE Reyes Ramos and H Mercado López, from Mexico; F Cárdenas, R Inga and A Santivañez from Peru. Thanks as well to all the reviewers, collaborators and coordinators in the participating hospitals. Acknowledgements to M del Mar Fernández, E Sierra and C Colomer, from the Ministry of Health and Social Protection of Spain, and to LE Velásquez, R Centeno, G Perdomo, H Rodríguez, G Alfaro, J Suárez, J Santacruz, F Hernández, A Experato, JM Pérez-Gallego, J Ramiro Cruz, H Montenegro and JL Di Fabio, from the Panamerican Health Organization (PAHO) and the PAHO Country Offices in Argentina, Colombia, Costa Rica, Peru and Mexico. Special acknowledgement to L Donaldson for his vision.
References
Footnotes
Funding JMAA, CAR and RLR have been granted in some occasions by the Fondo de Investigaciones Sanitarias (Health Research Fund) related to the Spanish Ministry of Health Social Policy and by the National Health System Quality Agency of the Spanish Ministry of Health and Social Policy.
Competing interests None.
Ethics approval Ethics approval was provided by the Pan American Health Organization Ethics Review Committee and the corresponding national ethics committees of the participating countries.
Provenance and peer review Not commissioned; externally peer reviewed.