Article Text
PDF
Abstract
Background Little is known of the incidence of adverse events in the paediatric intensive care unit (PICU). Perceived incidence may be dependent on data-collection methods.
Objective To determine the incidence of adverse events by voluntary reporting and systematic enquiry.
Methods Adverse events in PICU were recorded contemporaneously by systematic enquiry with bedside nurses and attending doctors, and compared with data submitted voluntarily to the hospital's quality and safety unit. Events were classified as insignificant, minor, moderate, major and catastrophic or lethal, and assigned origins as medical/surgical diagnosis or management, medical/surgical procedures, medication or miscellaneous.
Results Among 740 patients, 524 adverse events (mean 0.71 per patient) occurred in 193 patients (26.1%). Systematic enquiry detected 405 (80%) among 165 patients and were classified by one investigator as insignificant 30 (7%); minor 100 (25%); moderate 160 (37%); major 103(25%) and catastrophic 12 (3%). The coefficient of agreement (kappa) of severity between the two investigators was 0.82 (95% CI 0.78–0.87). Voluntary reporting detected 166 (32%) adverse events among 100 patients, of which 119 were undetected by systematic reporting. Forty-nine events (9%) were detected by both methods. The number and severity of events reported by the two methods were significantly different (p<0.0001). Voluntary reporting, mainly by nurses, did not capture major, severe or catastrophic events related to medical/surgical diagnosis or management.
Conclusions Neither voluntary reporting nor systematic enquiry captures all adverse events. While the two methods both capture some events, systematic reporting captures serious events, while voluntary reporting captures mainly insignificant and minor events.
- Adverse event
Statistics from Altmetric.com
Introduction
Hospitals are hazardous environments for patients. Retrospective reviews of medical records identified adverse events occurring at rates of 4–17% of all admissions in groups of hospitals worldwide,1–5 but information on the incidence of adverse events in individual paediatric institutions and paediatric intensive care units (PICU)6–8 is scant. A study of all hospitalisations in the two US states of Colorado and Utah, determined that 1% of all children sustained an adverse event, of which 60% were preventable.9
Since a vital component in a programme to ensure patient safety is recognition of the occurrence of adverse events, accurate data collection is essential, but this may be dependent upon the method of reporting. Traditionally, quality-assurance programmes rely upon voluntary reporting, but this has underestimated the true number of adverse events in disciplines of anaesthesia10 and neonatology.11
The purpose of this study was to determine the contemporaneous incidence of adverse events by systematic enquiry in a paediatric intensive care unit and to compare it with voluntary reporting.
Methods
The study was conducted in the paediatric intensive care unit (PICU) at the Royal Children's Hospital (RCH), Melbourne, Victoria, Australia. RCH is one of two state paediatric tertiary referral hospitals and houses the principal ICU. The state has a population of 6 million, of whom one million are children (<15 years).
The PICU has 15 high-acuity beds, is the national extracorporeal membrane oxygenation (ECMO) referral centre and operates a paediatric emergency retrieval service for the States of Victoria, Tasmania and southern New South Wales. The annual admissions for 2006–2007 were 1329, of whom 73% received mechanical ventilation via endotracheal intubation. The mortality was 4.36%, and average duration of admission 3.86 days.
All patients admitted to the PICU were studied on 176 days in the period from June 2006 to May 2007 when both investigators were available. The need for informed consent was waived by the hospital's human research ethics committee. The patients included all ages from infancy to 19 years and were representative of the different childhood illnesses/disorders.
The routine reporting process of adverse events in the hospital is conducted on a voluntary basis either by paper submission or on-line by ‘RiskMan,’ which is an incident-reporting system (Port Melbourne, Australia, 3207). It is available at all hours every day. ‘RiskMan’ requests details of the incident including those of the patient, identification of the informer, classification and contributing factors to the event. An option exists to complete the report anonymously.
The information sought under the term ‘adverse events’ related to harm defined as any unintended physical injury resulting from or contributed to by medical care or error including the absence of indicated medical treatment that required additional monitoring, treatment or hospitalisation or which resulted in death. The injury was considered harm, irrespective of whether or not it was considered preventable. ‘Near misses’ and complications associated with illness were excluded.
Systematic reporting was conducted in a stepwise fashion. This began with attendance by one investigator at the morning medical staff patient hand-over meeting. This was followed during the day (08:00–18:00) by conductance by both medical investigators of a ‘quality in care’ bedside ward round on week or weekend days, which involved the attending medical and nursing staff with additional requested input from the nurse manager and the senior medical staff on duty. The patient's case notes, medication and observation charts were also reviewed along with laboratory and diagnostic tests. An initial open-ended question to bedside nurses and attending doctors ‘Has any adverse event occurred with this patient?’ was followed by specific questions related to, for example, drug errors, delays in initiating treatment/procedures, need for additional surgeries, any care thought to be suboptimal or which resulted in additional investigations or a longer duration of hospital stay. Parental opinion as to perceived quality of care was also obtained when possible. A record was made of a repeat admission from the medical/surgical wards within a 24 h period of transfer as well as cancellation and delay in surgical procedures. Potential events were followed on a daily basis to determine if actual harm had resulted. Some patients required follow-up after they were discharged from the PICU to other wards or to their homes in order to determine the severity of an adverse event.
The causes of adverse events were classified as due to management decisions, medical or surgical procedures, medication errors or other therapeutic care.
The events considered harmful were assigned a category of consequence according to those published by Standards Australia, Risk Management Standard AS/NZS 4360:2004 which specifies five categories of patient outcome:
insignificant: no actual injury or increased level of care or length of stay;
minor: increased level of care, additional investigation or referral to another clinician or hospital service;
moderate: increased length of stay or additional operational procedure;
major: increased length of stay requiring significant medical or surgical intervention;
catastrophic: unexpected or preventable death or permanent disability.
In the category of ‘insignificant’ outcome, actual harm may not have been identifiable, even though an adverse event such as a drug dose error had actually occurred in contrast to a discarded ‘near miss’ event in which the error was detected before drug administration. The two investigators independently applied their own severity ratings to each systematically detected event, which were then compared to determine the inter-rater agreement by calculation of κ. The first investigator alone applied severity ratings to voluntary reported events. Outcomes captured by systematic enquiry and voluntary reporting and scored by the first investigator were compared with the Fisher exact test using Stata 9.0 (StataCorp, College Station, Texas).
Results
Systematic reporting
One hundred and sixty-five patients were determined by systematic enquiry to have sustained an adverse event. These patients had a total of 405 adverse events, with 96 having a single event, 69 having two or more, and 21 patients recording at least five or more. The impacts of these adverse events, classified by their outcome consequence according to the first investigator, were: insignificant 30 (7%), minor 100 (25%), moderate 160 (37%), major 103(25%) and catastrophic 12 (3%). The severity of adverse events was judged independently by the second investigator and compared with that of the first investigator. The coefficient of agreement (κ, κ;) between the two investigators was 0.82 (95% CIs 0.78 to 0.87). The origins of adverse events were medical/surgical management decisions 59%, medical/surgical procedures 28%, medication errors 11%, and miscellaneous 2%.
Voluntary reporting
Voluntary reporting captured 166 adverse events among 100 patients. The majority of voluntary reports (148, 89%) were by nurses, and the events were insignificant, minor or of moderate severity. Only four major events were reported, and these were by doctors (figure 1). The first investigator classified the severity of voluntarily reported events as: 33 (20%) insignificant; 117 (71%) minor; 12 (7%) as moderate; 4 (2%) and none as catastrophic. Of the 100 patients with voluntarily reported events, 37 had two or more events. Of these 100 patients, 28 had not been included in the systematic study, each had a single adverse event, and the 16 patients recorded by both methods had an additional 23 adverse events not detected in the systematic study. Voluntary reported events were medical/surgical management decisions 3%, medical/surgical procedures 53%, medication errors 42% and miscellaneous 2%.
Comparison of adverse events reported voluntarily and systematically.
Combined reporting
The combination of systematic and voluntary reporting yielded a total of 524 adverse events occurring among 193 in a total of 740 patients (26.1%), which is an average of 0.71 events per PICU patient. Systematic study captured 405 events (77%), whereas voluntary reporting captured 166 (32%), with 119 being undetected by systematic reporting. Forty-nine events (9% of total), were captured by both methods, of which 28 (57%) had insignificant, and 21 (43%) had minor outcomes. The quantity and severity of adverse events were significantly different between reporting methods (p<0.0001).
Discussion
This study revealed that adverse events are common in the paediatric intensive care environment and that they are captured to different extents by the methods of reporting. This is one of few studies which have determined the incidence of adverse events in the PICU setting. Adverse events have been studied systematically in adult but infrequently in paediatric institutions. The first study of adverse events among all admissions in 51 hospitals in New York State (The Harvard Medical Practice Study) identified a rate of 3.7%.1 Similar studies in Australia of 28 hospitals (The Quality in Australian Health Care Study) identified a rate of 16.6%,2 while a study of two hospitals in UK yielded a rate of 10.8%,3 of 13 hospitals in New Zealand 12.9%4 and of 20 hospitals in Canada 7.5%.5 Extrapolations to countries of origin estimated annual death tolls of 100 000 in the USA, which was the eighth leading cause of death in that country,12 while in Australia it was estimated at 18 0002 and in Canada at 9000–24 000 deaths.5
Only a few studies indicate the consequences of adverse events in entire paediatric institutions. Among these, the incidence of adverse events in children ≤15 years in the Harvard Medical Practice Study was 2.7% of hospitalisations. Another study of hospitalisations in the two States of Colorado and Utah determined that 1% of all children sustained an adverse event, of which 60% were preventable,9 and the incidence among adolescents in the study was 2.74%.13 In Australia, children (0–14 years) sustain adverse events in 10.8% of admissions, causing an additional average admission duration of 4.1 days.2 At our institution, adverse events occurred in 16.8% of admissions according to a study of selected medical records of selected patients who had died, unexpected admissions to PICU and specific doctor and nurse referrals.14
Adverse events occur more frequently in the adult intensive care unit setting, as shown by a study15 which found that 20.2% of patients sustained an adverse event, of which 55% were deemed non-preventable and 45% preventable, and of which 13% were life-threatening or fatal. Similarly, the PICU environment is risk-laden, but the incidence and severity of adverse events may be attributed in part to the method of study. The number of patients in our PICU sustaining an adverse event at 26% (0.7 events per patient) is higher than in another PICU at 8% (0.11 events per patient).8 The lower incidence in the latter may be explained by the absence of admission of patients of greater complexity, including those requiring solid organ transplantation, extracorporeal membrane oxygenation or difficult cardiac surgery. In other respects, however, the methodology in that study was similar to our study: it was prospective with evaluation of adverse events by physicians, and although the incidence of adverse events was smaller, a preponderance of the events were similarly classified as significant with 52% of children having a major outcome (including two direct deaths), 33% moderate and 16% minor outcomes. In contrast, in another PICU study conducted by doctors and nurses by retrospective chart review,7 the incidence of adverse events was very high, with 59% of patients sustaining an adverse event (1.96 events per patient), of which 78% of adverse events were classified as minor outcomes, 19% as moderate and 3% as serious outcomes with no deaths.
The high incidence of adverse events and the high outcome severity in this study (25% major, 3% catastrophic) and others may be related to the high acuity of illness and the invasive nature of medical intervention in the intensive care unit setting. Although we did not study factors causing or associated with adverse events, another study7 showed that adverse events were related to young age, length of stay and severity of illness, but another study8 observed that only length of stay was important.
Among all infants hospitalised with bronchiolitis, 10% sustained adverse events, but the incidence was 14 times higher among critically ill infants (68 vs 5 per 100 admissions).16
Recognition of adverse events and their severity are important steps in the process of prevention, but accurate identification of adverse events is essential. Although we observed that systematic reporting captured not only more adverse events but also more serious adverse events than voluntary reporting, we did not know the ‘gold standard’ or the most accurate method of reporting. Almost 30% of adverse events in this study were categorised by the medical investigators as having a major or catastrophic outcome, compared with 70% as insignificant, minor or moderate severity. This contrasts markedly not only with the fewer number but also the lesser severity of adverse events reported voluntarily. No event was classified by volunteer reporting as catastrophic, and only 9% were classified as being of moderate or major severity. The majority of voluntary reporting was by nurses and involved medication and procedural errors, whereas the systematic investigation, by the two medical investigators, also detected errors of diagnosis and management. A comparative study of adverse events reported voluntarily by nurses and doctors in another paediatric care unit environment also suggested under-reporting of less severe events by nurses.6
The differences in the incidences of adverse events reported by the different reporting methods may be due in part to different roles of the two reporting groups: systematic enquiry by two doctors versus voluntary reporting by numerous nurses. Although there is some overlap of roles, doctors focus more on diagnosing and treating patients, whereas nurses focus more on delivering care. Consequently, each group may be more adept at recognising and more comfortable in reporting events stemming from different practices. Despite the lower yield from voluntary reporting, it is nonetheless valuable because it is likely to encourage a sustained culture of improvement among all staff. Nonetheless, voluntary reporting underestimates the incidence and severity of adverse events in the PICU environment.
The difference in the incidence of adverse events by voluntary and systematic reporting in this study follows a trend observed in other disciplines. For example, anaesthetic-related adverse events were estimated at 7.7% by a systematic reporting method versus 1% using a voluntary self-reporting method, although outcomes were more severe in the self-reporting method.10 In the newborn intensive care unit setting, a systematic method using an NICU-focused trigger tool estimated the incidence of adverse events at 74 per 100 patients, but only 8% of these were identified with a voluntary-based method.11
Of all adverse events, medication errors have been subjected to most attention, not only because they are numerous and serious in general medical1 and paediatric practice17 18 but also perhaps because they are more easily identifiable and perhaps more amenable to prevention than other adverse events. In another study, two systems of reporting worked well in tandem to detect and prevent medication errors: voluntary reporting excelled at detecting administration errors, whereas computerised surveillance detected errors associated with high-risk medications and evolving clinical status.19 In our study, medication errors constituted a major proportion (42%) of adverse events reported voluntarily and a substantial proportion (11%) of systematic reporting. Although numerous strategies have been proposed to prevent paediatric medication errors,18 recommendations about reporting methods were not considered.
Apart from jepordising patient outcomes, the cost of adverse events should add to the impetus for prevention. In the State of Victoria, Australia, where at least one adverse event occurred in 6.88% of all patients in 45 hospitals, the duration of admission was prolonged by 10 days, the risk of in-hospital death was increased seven times, the average admission cost increased by A$6826, and in-patient hospital budget increased by 18.6%.17
A substantial number of adverse events are due to care below an accepted standard. In a medical and legal analysis of the results of the Harvard Medical Practice Study of 1991, of the adverse events which occurred in 3.7% of hospitalisations, 27.6% were deemed negligent,18 which we assume would likely fulfil all of the requirements of a tort of negligence. We declined to apply the same legal assessment to events recorded in this study, but suffice to say that many appeared preventable and may not have occurred if accepted practices had been followed. At least one event is the subject of current legal action.
Conclusions
The environment of the paediatric intensive care unit is hazardous for patients. Adverse events are common. Detection of these events is related to the method of reporting. Voluntary reporting is mainly conducted by nurses, is largely restricted to events of insignificant, minor or moderate consequences for the patient, and concerns medication errors. In contrast, systematic reporting by medical investigators captures events of more serious nature and is centred on diagnosis and management. It is recommended that systematic reporting of adverse events be adopted in addition to voluntary reporting by paediatric intensive care units. This would improve recognition of the nature and extent of adverse events as an important component in the development of preventive strategies.
Acknowledgments
We thank M Merriman, for data related to admission numbers during our study, K Dunn, for commentary on the manuscript, B Carter, for some of the statistical analyses, and numerous nurses and doctors who responded to our questions.
References
Footnotes
Competing interests None.
Ethics approval Ethics approval was provided by the Royal Children's Hospital's Human Research Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.