Introduction

Patient safety has achieved a high priority in many health care systems since the report by the Institute of Medicine in the USA [1] and publication in other countries [2, 3, 4, 5]. Patient safety is the attempt to assure that a course of medical treatment will proceed correctly and provide the best possible chance to achieve a desired outcome. Patient safety is therefore an essential component in the practice of intensive care medicine. However, it seems that the complexity of processes and medical conditions dealt with in intensive care units (ICUs) makes the system vulnerable and prone to error [6]. Because the close monitoring and supervision typical of ICU environments offer an ideal opportunity to detect errors and their contributing factors, there is a growing body of literature on several types of unintended events in ICUs [7, 8, 9]. Nearly all existing reports are based on observations in single ICUs or in relatively small national samples [10, 11, 12, 13]. Consequently, two important questions have not yet been answered: Is there a general pattern of error susceptibility in ICUs? If so, what is the prevalence of important types of errors in this setting?

We conducted a prospective, observational, multinational study on the prevalence of selected unintended events in intensive care medicine. A cross-sectional design with a 24-h observation period was chosen to ensure a concentrated effort on data collection. In addition, we assessed the impact of several ICU- and patient-related factors, such as workload of medical personnel and severity of illness, on the frequency and types of selected unintended events.

Methods

The Sentinel Events Evaluation (SEE) study was conducted by the Research Group on Quality Improvement of the European Society of Intensive Care Medicine (ESICM) in association with the Section on Health Services Research and Outcome. The study was endorsed by the European Critical Care Research Network. After an announcement and call for participants, 280 ICUs registered for the study by December 2003. During the registration process, the date of the study — January 21, 2004 — with its limited 24-h observation period was communicated to all participants. Ultimately, 220 ICUs from 29 countries and four continents participated.

Because the study was observational and no additional interventions were performed, the need for informed consent was waived by the institutional review board. Each ICU, however, was made responsible for obtaining local permissions as necessary.

Definition and assessment of sentinel events

A sentinel event was defined as an occurrence that harmed or could have harmed a patient. We looked at selected unintended events within the following categories: (1) medication (wrong dose, drug, route); (2) airway (unplanned extubation, artificial airway obstruction, cuff leakage prompting reintubation); (3) indwelling lines (intravascular cannulas and the attached fluid delivery sets), catheters, probes and drains (unplanned dislodgement, inappropriate disconnection); (4) equipment failure (infusion devices, ventilator and accessories, renal replacement devices, power and oxygen supply); and (5) alarms (inappropriate turn-off).

All nurses and physicians on duty in the participating ICUs during the 24-h study period were requested to answer a questionnaire available at the bedside of each patient. The questionnaire asked in a structured form whether and at which time any of the selected sentinel events had happened. In addition the utilisation of devices that could lead to the occurrence of selected sentinel events (e.g. use of several lines, drains, catheters) was assessed (see electronic supplementary material). An ICU coordinator in each participating ICU was responsible for the briefing of the ICU team, data collection and transmission. His or her obligations included the provision of information regarding ICU characteristics and classification of severity of illness and medical personnel workload for every patient. Data acquisition and reporting were anonymous for both patients and medical personnel.

Patient sample

The study included all patients staying in the participating ICUs, including patients admitted or discharged, during the 24-h study period.

Patient-related factors

For individual patients, the medical personnel recorded age, sex, time of ICU admission and discharge, and frequency, characteristics and time of the predefined unintended events. To facilitate reporting without the fear of blame there was no collection of any information regarding the outcome of unintended events.

To determine the severity of illness in every patient, the items necessary for calculating the Sequential Organ Failure Assessment (SOFA) score (to quantify the presence and degree of organ dysfunction/failure and consequently provide a surrogate for severity of illness) were collected [14], and all items for the Nine Equivalents of Nursing Manpower Use Score (NEMS) were also recorded (as a surrogate marker for the amount of nursing workload use and consequently of the level of care provided to each patient) [15]. The duration of each patient's stay in the ICU before and during the 24-h observation period was calculated from the time of admission to the time of discharge and is reported as “ICU length of stay before observation” (days) and “risk time” (hours), respectively.

ICU-related factors

For every ICU the following characteristics were recorded: hospital size, ICU type and size, shift schedule for nurses and physicians, number of nurses and physicians appointed to each specific shift, number of occupied and free beds in each shift, maximum number of patients in each shift and number of admitted and discharged patients in each shift. These data allowed us to calculate occupancy rate, patient-to-nurse ratio and patient-to-physician ratio for each shift and each ICU. Information about any existing system for formal critical incident reporting was also recorded.

Database and data collection

Patient data were recorded using a dedicated project website and online data collection software. The website contained (1) all documentation, including detailed definitions of all data fields and events (see electronic supplementary material); (2) data collection sheets available for download; and (3) access to the study database with its data entry forms.

To ensure consistent quality of the data in ICUs from different participating countries, the questionnaires were available in six languages: English, German, Italian, French, Portuguese, and Spanish (see electronic supplementary material).

Data collected from each ICU were entered into the study database by the appointed ICU coordinator. ICU coordinators who could not use the online data entry forms (e.g. because of a poor or nonexistent Internet connection) were allowed to submit paper data sheets to the project center (n = 12 ICUs). These data were entered into the online database system by project personnel.

During data entry by ICU coordinators or project personnel, plausibility and completeness checks were performed online. Each variable was defined in detail before the start of data collection, and the definitions were available in both paper and electronic form. To facilitate plausibility checking, each variable was assigned a range of probable values and a range of possible values (storage range).

Data collection started at the beginning of the nurses' day shift. The study period was designed to overlap a second day (but not to exceed 24 h) to be sure that data acquisition in each ICU included at least one day shift and one night shift.

Statistical analysis

Statistical analysis was performed using the SAS system, version 8.02 (SAS Institute, Cary, NC). Either the number of all patients in the final study sample, the number of patients exposed to a specific item, the number of all ICUs contributing patients to the final study sample, or the number of patient days (calculated as total hours at risk for all patients, divided by 24) served as denominator. The number of patients suffering an event or the number of events in total or in different categories served as numerator.

We calculated odds ratios for the risk of experiencing an unintended event at least once during the observation period. In a first step, multilevel logistic regression with ICU as a random component was applied separately per factor using linear and, if appropriate, quadratic terms to investigate the influence of all factors assessed as patient related. The same univariate model was applied to assess the influence of all collected ICU-related factors. Smoothed distribution graphs for the event probability were used to clarify the form of the univariate dependencies. Finally, a multiple logistic regression with backward elimination and ICU as a random component was performed. We included influence variables significant in the univariate logistic regression, eliminating a few of these variables due to high correlations with the others.

Unless otherwise specified, results are expressed as median (1st – 3rd quartile). All P values are based on two-tailed tests of significance. A P value of < 0.05 was considered significant.

Results

ICU and patient sample

After exclusion of 173 patients aged < 18 years the final study sample consisted of 1,913 adult patients (61% men) from 205 ICUs and 29 countries (Table 1). The mean age was 62.3 (± 16.3 standard deviation; range 18–99) years. Median ICU length of stay before observation was 2.8 days (quartiles 0.5–11.2). The median SOFA score was 4 (quartiles 2–7). The observed risk time (actual time of a patient within the ICU during the observation period) ranged from 0.5 to 24 h, median 24.0 (quartiles 15.0–24.0). The median NEMS was 27 (quartiles 18–38).

Table 1 Final study sample: participating ICUs and numbers of patients by country
Full size table

The majority of the 205 ICUs in the final study sample were of a mixed type (n = 143); the remaining ICUs were described as surgical (n = 24), medical (n = 19), trauma (n = 9) or other (n = 8). The ICUs were located in hospitals with < 300 beds (20%), 300–600 beds (44%) or > 600 beds (36%). The ICUs had a capacity of < 7 beds (19%), 7–12 beds (58%) or > 12 beds (23%). Most ICUs were organised with three nurse shifts (70%) and two (52%) or three (48%) physician shifts per day. The median patient-to-nurse ratio ranged from 1.3 (shift 1) to 2.0 (shift 3), and the median patient-to-physician ratio ranged from 3.0 (shift 1) to 6.0 (shift 3). In 39% of the ICUs included in the final study sample, an existing system for formal critical incident reporting was in use.

Occurrence and type of sentinel events

In the 1,913 patients, 584 sentinel events affecting 391 patients were reported for the 24-hour study period. Most (1,522) patients experienced no event, 268 patients experienced only 1 event, and 123 patients experienced > 1 event. Fifty-five (27%) of the 205 ICUs reported no sentinel events.

In total, 38.8 (95% confidence interval 34.7–42.9) events per 100 patient days were observed. Table 2 gives the corresponding numbers for all event categories. There was no noticeable difference in the number of events per 100 patient days between ICUs with or without an existing incident reporting system in use.

Table 2 Observed rates of sentinel events
Full size table

Within the five selected categories, the most frequent events were related to lines, catheters and drains, and occurred in 158 patients (Table 3). Events associated with the prescription (74 patients) or administration (62 patients) of drugs were the second most frequent observation: drug administration errors included a wrong drug in 15 patients, a wrong dose in 38 patients and a wrong route in 9 patients. The most frequent equipment failure was related to infusion devices (n = 37), followed by failure or dysfunction of ventilators (n = 20; occurred in 1.9% of all patients receiving mechanical ventilation) or of renal replacement devices (n = 12; occurred in 8.5% of all patients receiving renal replacement therapies). Failure of power or oxygen supply at the patient's bedside was reported in 20 and 9 cases, respectively. An unplanned extubation occurred in 12 of the 1,627 patients with an endotracheal tube or cannula in use, and an artificial airway obstruction or cuff leakage occurred in 23 and 13 patients, respectively. An inappropriate turn-off of alarms was reported on 17 occasions. Only one of these 17 occurrences was related to a corresponding event from another category (failure of oxygen supply).

Table 3 Sentinel events related to lines, catheters and drains
Full size table

Predictors of sentinel events

An univariate analysis showed that a higher severity of illness, a higher level of care, a longer length of stay in the ICU before observation, and a longer duration of exposure (risk time) were associated with elevated odds for experiencing a sentinel event (Table 4).

Table 4 Patient-related variables associated with elevated odds for experiencing a sentinel event (univariate analysis)
Full size table

With respect to ICU-related variables, the univariate analysis showed an association of trauma ICUs with lower odds for the occurrence of sentinel events (odds ratio 0.47, 95% confidence interval 0.22–1.00; P = 0.04). The patient-to-nurse ratio exhibited a slight nonlinear influence (because of that nonlinearity, it has no overall odds ratio; linear term P = 0.02, quadratic term P = 0.006), as did the number of admissions and discharges (linear term P = 0.04, quadratic term P = 0.003). No other ICU-related variables were significantly associated with the occurrence of sentinel events.

In a stepwise multiple logistic regression analysis with ICUs as a random factor and backward elimination, one ICU-related and three patient-related variables remained in the final model (Table 5). Of note, this advanced statistical model explains only 11% of the observed variations, of which 5% are contributed by the remaining variability on ICU level.

Table 5 Stepwise multiple logistic regression: final model
Full size table

Discussion

The results of the SEE multinational study confirm previous findings from single institutions showing that patients in ICUs are frequently exposed to errors [6, 7, 8, 9, 10, 11, 12]. For five selected categories of intensive care management, we found a total prevalence of 38.8 unintended events per 100 patient days. Considering that this number, derived from self-reports, may underestimate the frequency of errors, these results may be of even greater clinical relevance. ICUs have been developed to provide optimal care for patients at risk from several medical conditions, but it turns out that this particular clinical environment, like other hospital areas, carries its own risks [16]. The SEE study demonstrates that these weak points in patient safety are not attributable to suboptimal care in a few individual ICUs but represent a common pattern. Considering the complexity of highly developed intensive care, this is not surprising. In their landmark study in 1995, Donchin et al. reported an average of 178 daily activities in intensive care patients; this number may have increased since that report was released [6].

Furthermore, intensive care is characterised by a complex course of interaction among several medical specialties [17]. One important example of this interaction is the process of prescribing and administering drugs. In the SEE study we found a nearly equal prevalence of errors in prescription and administration of drugs, resulting in 10.5 events per 100 patient days. This observation is in line with reports from several single ICUs [18, 19, 20, 21, 22] and highlights an area of risk for potential patient harm. In one study, 20% of medication errors at the administration stage were considered potentially life-threatening [18].

Another area with a high potential for patient harm is the management of artificial airways [23, 24]. In addition to possible immediately life-threatening complications, at least one study has shown that unplanned extubations are associated with prolonged mechanical ventilation, longer ICU and hospital stay and increased need for chronic care [25]. The authors reported 1.6 unplanned extubations per 100 days of intubation. This number corresponds fairly well to our event rate of 0.7% unplanned extubations.

The most frequent occurrences were related to the unplanned dislodgement or inappropriate disconnection of lines, drains and catheters. Both types of unintended events have the potential for considerable harm [26]. Although the unintended removal of a Foley catheter or an enteral nutrition probe might be seen as a small misadventure, even these minor events could lead to urinary tract bleeding or aspiration of gastric content. In contrast, the risks associated with the dislodgement and disconnection of dialysis catheters are evident, and a prevalence of 3.8% and 1.9%, respectively, in our study should be noted with concern [27, 28].

Intensive care depends on the stable functioning of technical equipment. Our results show, for the first time in a large, multinational sample of ICUs, that technical failures frequently lead to events that compromise patient safety. This conclusion is in keeping with an analysis of 1,004 reports to French authorities of incidents involving medical devices in anaesthesia and intensive care that revealed 12 deaths attributable to the failure of a device [29].

As would be expected, univariate and multivariate analysis in our study showed that patients who are more severely ill and who receive a higher level of care are more likely to experience an unintended event. This finding directly reflects the complexity of care of these patients. Less clear was the analysis of ICU-related factors. Our observational study did not demonstrate a noticeable impact of the observed system-based factors on the occurrence of errors. The only significant variable, “patient-to-nurse ratio”, had a small effect with a surprising tendency for lower odds at low and high levels. A small range of the median patient-to-nurse ratio (1.3–2.0) and complex interactions with other ICU-related or unmeasured factors (e.g. sedation levels) as well as hidden causes may have contributed to this finding. Even more, it represents only a negligible part of the variation (0.6%) and might be a chance finding due to multiple testing.

In our multivariate model, the most pronounced effect was seen when ICUs were included as random components. This effect should be noted, but this sophisticated statistical model has little explanatory power. One reason might be that components that have been shown to influence the occurrence of errors, such as variations in ICU organisation [20], different formats of communication [30] and different cultures of communication [6] are difficult to measure and were outside the scope of our 24-h observation. Obviously another taxonomy [31] of contributing factors would have identified different contextual information. But even a simple chronological plot of the frequency of events (Fig. 1) reveals a pattern that is probably associated with several courses of action (e.g. rounds, shift changes) during daily ICU operation. Donchin et al. reported a similar diurnal distribution of critical incidents corresponding to physicians' and nurses' morning peak activities and the times of the nurses' shift changes [6].

Fig. 1
figure 1

Chronological distribution of sentinel events

Full size image

Other limitations of our study relate mainly to methodological questions. First, in order to facilitate reporting without any fear of blame we did not collect information regarding actual patient harm. Therefore it is not possible to draw any conclusions about the impact of unintended events on patients outcome. Second, our study focused on events related to errors of commission and did not record errors of omission. Third, we did not collect data about the use and impact of electronic medication prescribing systems, but with its growing propagation this technology becomes an area for further research. Finally, it is important to acknowledge several problems in measuring errors. Different formats of data collection like chart review or incident reporting will lead to different findings [32]. Using external observers might be considered as “gold standard” but is heavily resource consuming and implements the bias of different behaviour under observation (Hawthorne effect). In contrast, as shown by several authors, a self-reporting method carries the risk of under-reporting [6, 33, 34]. Because we used a facilitated reporting by a structured questionnaire for the occurrence of selected events in a short period of time, we are confident that this problem was minimised. Finally, in our surveillance the possibility of volunteer bias needs to be considered for the ICUs electing to join the study.

Results from the SEE study suggest an urgent need for ICUs to improve the safety of medication prescription and administration and the appropriate maintenance of indwelling lines, catheters, drains and airways, as well as to improve the development of error-free, stable technical equipment. Apart from the findings we have reported, the most remarkable aspect of the SEE study was the eagerness of a large number of ICU personnel worldwide to participate. The intensive care community has proved that patient safety is recognised as a top-priority issue and demonstrated that open communication is the starting point for addressing flaws in our current system of intensive care.