Article Text
PDF
Abstract
Introduction Although paediatric patients have an increased risk for adverse drug events, few detection methodologies target this population. To utilise computerised adverse event surveillance, specialised trigger rules are required to accommodate the unique needs of children. The aim was to develop new, tailored rules sustainable for review and robust enough to support aggregate event rate monitoring.
Methods The authors utilised a voluntary staff incident-reporting system, lab values and physician insight to design trigger rules. During Phase 1, problem areas were identified by reviewing 5 years of paediatric voluntary incident reports. Based on these findings, historical lab electrolyte values were analysed to devise critical value thresholds. This evidence informed Phase 2 rule development. For 3 months, surveillance alerts were evaluated for occurrence of adverse drug events.
Results In Phase 1, replacement preparations and total parenteral nutrition comprised the majority (36.6%) of adverse drug events in 353 paediatric patients. During Phase 2, nine new trigger rules produced 225 alerts in 103 paediatric inpatients. Of these, 14 adverse drug events were found by the paediatric hypoglycaemia rule, but all other electrolyte trigger rules were ineffective. Compared with the adult-focused hypoglycaemia rule, the new, tailored version increased the paediatric event detection rate from 0.43 to 1.51 events per 1000 patient days.
Conclusions Relying solely on absolute lab values to detect electrolyte-related adverse drug events did not meet our goals. Use of compound rule logic improved detection of hypoglycaemia. More success may be found in designing real-time rules that leverage lab trends and additional clinical information.
- Paediatrics
- adverse drug event
- computerised surveillance
- trigger tool
- information technology
- medication error
This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
Statistics from Altmetric.com
- Paediatrics
- adverse drug event
- computerised surveillance
- trigger tool
- information technology
- medication error
Introduction
Compared with adults, less information is available on the epidemiology of paediatric adverse drug events (ADEs), although they are more likely to occur.1 ADE rates ranging from 6.6 to 15.7 events per 1000 patient days have been reported for paediatric inpatients.2–4 Guiding agencies have suggested adoption of health information technology (HIT) tools to reduce medical errors, but the lack of paediatric-specific information to develop standards remains a barrier.1
Duke University Hospital (DUH) employs two ADE detection systems: a voluntary safety reporting system (SRS) and computerised ADE surveillance (ADE-S).5–7 SRS is a web application where staff report safety incidents, whereas surveillance applies trigger logic to medical records and notifies of potential ADEs.5 ,6 In paediatrics, ADE-S did not outperform SRS in ADE detection as it does in adults.5 ,6 ,8–10 The ADE detection rate by surveillance is over four times that of paediatrics, despite the expected higher incidence of events.5 ,6 Paediatric HIT products should rely on independent data and not extrapolation from adults.1 This deficiency, as well as the low paediatric ADE detection rate using adult-focused triggers, clearly demonstrates a need for rules tailored to paediatric care. Our goal was to devise paediatric-focused trigger rules that are both sustainable at DUH and productive enough to serve as a longitudinal safety indicator. To meet these objectives, trigger rules must have the following design criteria: (1) objective evidence of harm inferable from the medical record, (2) reviewer time investment justified by alert volume and true positive rate, and (3) potential to generate sufficient ADE data for monthly rate trending. We retooled surveillance using insight from voluntary reporting, lab results and guidance from practising paediatricians.
Methods
Study design
We performed a retrospective, cross-sectional study at DUH, a tertiary care teaching hospital having seven paediatric inpatient units (three intensive-care units (ICUs), two general care, two transitional care). We defined an ADE as an ‘injury resulting from the use of a drug.’11 Phase 1 entailed a three-pronged approach for rule development: review of historical ADEs, analysis of lab data and paediatrician consultation. New surveillance rules were implemented in Phase 2. This study was approved by the Duke University Health System Institutional Review Board.
Phase 1
Medication-related voluntary reports to SRS are routinely investigated by clinical pharmacists and scored for severity (table 1).5 Many hospitals turn to voluntary reporting to find emerging risk areas12–14 despite reporter bias in the data.15 ,16 We reviewed inpatient paediatric (age <18) ADEs reported to SRS between 12 January 2002 and 12 September 2007, and grouped them by primary medication class. We examined classes with the greatest event volume, not the highest ADE rate, since voluntary reporting is not a quantitative metric.
Duke University Hospital 7-point severity scoring system for patient harm
Given that electrolyte preparations and total parenteral nutrition/lipids (TPN/lipids) were primary causes of ADEs, we gathered paediatric lab results for ionised calcium, chloride, magnesium, potassium, sodium and triglycerides from 1 August 2005 to 31 August 2007 to pinpoint critical values for trigger logic. Total bilirubin was examined as well, given concern regarding neonate monitoring.17 We plotted histograms showing the distribution of lab value results (figure 1). As with development of any alerting functionality, balance between event detection and alert volume is necessary to ensure sustainability. For most labs, the upper and lower 0.5% of distributions defined critical values for rule development. For ionised calcium, the upper 2.5% was selected as the critical value given the shape of the distribution. Although the upper 0.5 percentile for triglycerides was 1376 mg/dl, this threshold would generate few triggers each month, which defies our design requirement for trigger volume. Consequently, we sought guidance, and 500 mg/dl was suggested as a threshold by the clinical administrator of SRS, who is also a practising neonatologist at DUH.
Histograms showing the distribution of paediatric lab test results. The distributions of paediatric lab test values for (a) potassium, (b) ionised calcium, (c) chloride and (d) magnesium were examined over a 2-year time period. Chloride, triglycerides and total bilirubin were examined but are not shown. In each graph, the vertical axis comprises the counts of lab results. The arrows represent the thresholds for the upper or lower 0.5% of all test results (upper 2.5% for calcium, B). These values were considered extreme outliers and guided trigger rule logic. Accepted reference values for each test were dependent on age, but the most common ranges are shown.
An additional rule was developed given a known issue with ADE-S detection of hypoglycaemia. The existing rule was based on the adult protocol to administer dextrose 50% as an antidote to low blood glucose (BG). However, most paediatric protocols do not call for this drug. The paediatric hypoglycaemia-related ADE rate at DUH is only 0.08 events per 1000 patient days compared with 4.47 events per 1000 patient days in adults.5 ,6 Therefore, we created a paediatric rule to alert for BG<50 mg/dl concomitant with an insulin order. We did not include oral hypoglycaemic agents in this logic, since nearly 90% of DUH hypoglycaemic events are due to insulin.
Phase 2
The DUH ADE-S system has been described previously.5 ,6 ,10 ,18 Each evening ADE-S reviews the electronic records of all DUH inpatients to identify potential ADEs. Clinical pharmacists perform a chart review to confirm ADE occurrence (ie, documented patient harm). Events are scored by three pharmacist reviewers (κ≥0.88) for severity (table 1)5 and causality.19 Paediatric-specific rules were evaluated between 27 November 2007 and 12 February 2008. Fifteen days were excluded due to reviewer unavailability.
Data analysis
Phase 1 lab data histograms were created using JMP 7.0 (SAS Institute, Cary, North Carolina). Phase 2 analysis included all alerts for patients discharged from a paediatric location. Positive predictive values (PPVs) were calculated as the percentage of alerts deemed ADEs subsequent to evaluation. Rates were calculated as ADEs per 1000 patient days. The Fisher exact test was used to seek significant differences in categorical data. CIs were calculated using the binomial approximation. Where appropriate, the ADE yields of the new and old trigger rules were cross-compared.
Results
Phase 1: Review of historical ADE data and critical lab values
SRS documents 385 medication-related ADEs occurring in 353 paediatric patients from 12 January 2002 to 12 September 2007 (table 2). Electrolyte preparations and TPN/lipids account for 24.4% (94/385) and 12.2% (47/385) of events, respectively. Representative cases include an incident where dispensing of dextrose-containing intravenous fluids was delayed, resulting in a BG of 27 mg/dl and administration of an interventional glucose bolus to reverse hypoglycaemia. The American Diabetes Association holds that hypoglycaemia, regardless of symptoms, harms the patient.20 Another patient developed premature atrial contractions from hyperkalaemia due to failure to discontinue potassium-containing intravenous fluids. This event was scored a severity of 3, since transient adverse effects occurred, and corrective therapy was used to lower the potassium level.
Inpatient paediatric adverse drug events found by voluntary reporting (n=385)
From 1 August 2005 to 31 August 2007, over 450 000 lab results for paediatric inpatients were available for ionised calcium, chloride, magnesium, potassium, sodium, total bilirubin and triglycerides. Figure 1 shows histograms for these results as well as the outlier values selected.
Phase 2: Implementation of paediatric-specific ADE-S trigger rules
Development of paediatric-specific trigger rules (table 3) was guided by Phase 1 results. Only two of the original adult-designed rules involve electrolytes: dextrose 50% with a BG <50 mg/dl (A1) and an order for polystyrene, a reversal agent for elevated potassium levels (A2). Between 27 November 2007 and 12 February 2008, DUH had 796 paediatric admissions and 9270 patient days across the general care (41% of patient days), ICU (53% of patient days) and transitional care (6% of patient days) environments. These visits produced 225 trigger alerts (rules P1–P9) for 103 patients (table 3). Fourteen alerts from the hypoglycaemia rule (P9) were deemed ADEs following chart review (PPV=6.2%). The electrolyte-based rules did not detect ADEs, as no objective evidence of harm was documented.
Trigger rules employed in paediatric adverse drug event (ADE) computerised surveillance
Both the adult- (A1) and paediatric-focused (P9) rules for hypoglycaemia were active to evaluate differential ADE capture. A total of 30 triggers fired on paediatric units from these two rules (table 3). Some alerts were not deemed ADEs due to erroneous BG readings or undocumented insulin administration despite an active order. The PPVs for both P9 and A1 were similar (60.8% and 57.1%), yet a greater number of ADEs were detected with the paediatric rule (1.51 vs 0.43 events per 1000 patient days, p=0.03). All ADEs captured by the adult rule were detected using the paediatric-focused logic.
Discussion
Our goal was to combine clinical evidence and provider insight to develop a sustainable, paediatric-relevant surveillance model that yields trendable data for ADE monitoring. A review of voluntarily reported safety incidents revealed electrolyte and TPN/lipid preparations underlie many ADEs. No rule targeting lab abnormalities detected ADEs in this study, although we improved hypoglycaemia ADE capture.
Our results are consistent with other paediatric trigger development efforts. Sharek and colleagues developed a neonatal intensive care unit (NICU)-focused manual tool for adverse event discovery.21 They included rule logic for rising creatinine (PPV=11%) and an ‘abnormal electrolytes’ rule with a PPV of 8%. Since this latter rule is not retained in the trigger tool online (http://www.chca.com), we infer it was ineffective. A second trigger tool for paediatrics was deployed across 12 children's hospitals,4 and compound rule logic produced better PPVs. Although a single-value hyperglycaemia trigger resulted in a PPV of only 0.60%, combining absolute lab values with age logic to detect hyperkalaemia or nephrotoxicity resulted in PPVs of 3.57% and 3.85%. A recent study modifying a surveillance system for paediatrics had limited success with ADE detection using hypokalaemia and hypomagnesaemia rules.22 The authors also report a PPV of 0.08 for hyperkalaemia combined with age logic (potassium >6.0 mEq/l and age >1 year), whereas our PPV for potassium >7.0 mEq/l was 0.
These studies indicate that the use of compound rule logic improves paediatric ADE discovery. At DUH, the issue with using absolute lab values for ADE-S may lie in how they are drawn. For example, NICU heel sticks are used to obtain small blood samples, yet they frequently haemolyse, resulting in elevated potassium values. In the case of elevated lipid levels, the nurse may have drawn the sample from the line infusing lipids. Such details go undocumented, meaning relying on lab values retrospectively to suggest that ADE occurrence is ineffective for our goals. Options to improve rules using compound logic include reasoning over lab value changes (eg, rising bilirubin), since utilising more than two values may bypass unwanted alerts from a single, aberrant result. Additionally, concomitant active drug orders and lab results (eg, use of electrolyte preparations, potassium supplements, or potassium-sparing diuretics and hyperkalaemia) may be of value. Institutions with text scanning capabilities may be able to combine keywords from the medical record relevant to a lab result (eg, altered mental status and the presence of hyponatremia). Since ADE-S runs in a batch process, we are evaluating logistics for real-time alerting of concerning lab trends. Takata and colleagues reported two-thirds of their ADEs were due to lapses in monitoring medications, including assessment of lab results.4 This series of strategies may reduce false-positive alerts and improve ADE detection. Although lowering lab value thresholds may capture more ADEs, the expected increase in alert volume without improving the PPV would be a barrier to sustainable review—a central goal of this study.
There are several limitations to this work. DUH has a large paediatric ICU population (53% of study patient days), which accounts for 61% of trigger alerts. This may limit the generalisability of our findings to hospitals with a similar composition, yet these are likely the same facilities that would have IT resources to implement a computerised surveillance programme. A second limitation is that rules were evaluated over 2.5 months, which may have under-represented detection of rare ADEs. The authors felt that 2.5 months was adequate to evaluate alert utility given the goals of (1) sustainability and (2) data capture for aggregate ADE rate monitoring. Though not useful for our purposes, other organisations may have more success with these rules if able to sustain a longer study period. However, a recent 6-month study with over 40 000 patient days similarly reported low PPVs for electrolyte-based rules (except hypokalaemia and hypomagnesaemia).22 They also noted the need to modify or eliminate low-performing rules to balance reviewer effort and event discovery despite the possibility of rare ADE detection.22 A final limitation to any surveillance approach is its reliance on objective data in the medical record to confirm ADE occurrence. Near misses cannot be captured if errors did not reach the patient.
Conclusion
Although computerised surveillance effectively detects ADEs, developing paediatric rules based solely on critical lab values did not meet our goals. However, enhancement of a hypoglycaemia trigger rule significantly increased our detection rate. Investigators should be encouraged not only to report their successes in this area but also to describe ineffective approaches. The sustainability of trigger evaluation should be a consideration for any rule design. Development of compound rule logic may be a worthwhile approach.
What is already known about this topic
Paediatric patients are at high risk for adverse drug events in the hospital setting. The application of novel detection strategies, such as computerised surveillance, however, is underexplored in this fragile population. To date, inpatient paediatric trigger tool rules have mainly been extrapolated from logic designed for adults.
What this study adds
We used a three-pronged approach to develop new trigger rules specifically for hospitalised paediatric patients: analysis of a voluntary safety reporting database, evidence from historical lab values and concerns from practising paediatricians. We share our experiences with these new rules and evaluate their efficacy when added to a sustained computerised surveillance safety programme at a large academic medical centre. Our lessons learnt may aid other organisations seeking to create paediatric-targeted rule sets that support their safety event-detection programmes.
Acknowledgments
The authors thank S Pladna for technical assistance in programming trigger rules, D Leonard for safety reporting system assistance, and M Wu and H Shang for providing data warehouse extracts.
References
Footnotes
Funding This study was supported by grant no 5UC1HS014882-03 from the Agency for Healthcare Research and Quality, National Institutes of Health.
Competing interests None.
Ethics approval Ethics approval was provided by the Duke University Health System Institutional Review Board (Durham, NC).
Provenance and peer review Not commissioned; externally peer reviewed.