Article Text
PDF
Statistics from Altmetric.com
The emergency department (ED) abounds with hazards that might lead to adverse events (AEs). Clinicians must make frequent, high-consequence decisions with high diagnostic uncertainty as part of ad hoc teams taking care of patients not generally known to them.1 This occurs in the context of a sometimes chaotic environment with frequent interruptions, handoffs and shift changes. As a result, the ED tends to magnify cognitive biases that can result in AEs—particularly diagnostic AEs.2 This is especially problematic for children who account for 23 million ED visits in the USA annually and are particularly vulnerable to AEs. Children are developmentally less able to communicate specific symptoms and are evaluated in EDs with variable readiness to care for them.3 Yet, little is known about the occurrence and nature of AEs in paediatric ED patients.
In this issue of BMJ Quality & Safety, Plint and colleagues4 sought to address this gap. They conducted a prospective observational study of children visiting nine Canadian paediatric EDs to determine whether an AE occurred within 3 weeks of a visit.5 AEs were defined as ‘any event resulting in unintended patient harm that was related to the care provided rather than to an underlying medical condition’. They found that 3% of visits were associated with an AE, of which 77% were judged to be preventable. The most common AEs related to management, diagnosis and medication. Although the large majority of AEs were minor, half prompted the need for subsequent care. Risk factors associated with a preventable AE were increasing age, history of chronic condition, increasing time to initial ED assessment and evaluation in the acute care area (rather than the ambulatory area).
This study illuminates an underexplored area in paediatric emergency care: given its inherent hazards, how often do AEs happen? Plint and colleagues improved on their prior single-centre study,6 providing more precise and stable estimates of AE rates in this population while bolstering external validity, since estimates were consistent across the nine centres. Given that most AEs were preventable, this study also provides a clear basis for robust quality improvement efforts that have potential to reduce such AEs.
The key strengths of this study are its multicentre nature with high enrolment numbers, and impressive and meticulous efforts to follow patients forward from their ED visit, resulting in a robust identification of AEs. Investigators selected an unbiased sample of ED shifts, examined associated hospital records of admitted patients using the Canadian Paediatric Trigger Tool and made every effort to contact all patients to identify AEs. While it would be impossible to avoid recall bias when interviewing families about their past experiences, investigators minimised this risk through a multipronged, preplanned strategy of AE identification and vetting. This study’s findings therefore represent an important advance in our understanding of AEs occurring in paediatric emergency care.
As in any study, there were a number of limitations, including some that are common to most studies of AEs. Although the data are from 2014 to 2015, this is unlikely to affect the types of AEs detected or diminish the study’s significance. The types of conditions seen and potential events that occurred in the ED (such as peripheral intravenous catheter infiltrations, bleeding events, delayed diagnoses) are unlikely to have significantly changed since these data were collected. The main limitations involve three key issues: (1) methodological approaches affecting AE rates, (2) inherent subjectivity in AE determination, and (3) generalisability of paediatric ED AE rates to all EDs where children receive care.
Methodological approaches affecting estimates of AE rates
This study estimated the paediatric ED AE rate to be 3% of visits. In adult cohorts, AE reviews based on a broader trigger methodology have identified AE rates of 8%.7 One meta-analysis of mixed paediatric and adult populations by Stang et al found the range of AEs attributable to ED care to be 0.2%–6.0%.8 The wide range of AE rates in earlier studies highlights how decisions about the method of detection of AEs may influence reported rates. For example, the lower bound of 0.2% reported in Stang’s analysis was based on voluntary error reports, which are known to severely underestimate AEs.
There are reasons to believe that the true prevalence of AEs among children in the ED may be higher than this study reports. First, events under consideration had to be judged as at least 50% likely to be AEs to be included as such. This is a higher standard than the ‘possible AE’ standard, in which determination of an AE hinges on whether some action or omission ‘may have contributed to’ an AE. This standard is used in detecting all-cause harm, in part to avoid subjectivity.9 The higher 50% standard used in Plint and colleagues’ study requires judgements as to whether, for example, hypotension in a patient with sepsis who received morphine was possibly an AE due to the morphine or was attributable to their underlying condition. Second, because subsequent care was a key means of identifying AEs in Plint and colleagues’ study, events that did not prompt subsequent care would only have been included if families were aware of and reported them. Thus, minor AEs could have been missed.
Conversely, other methodological decisions in this study might have increased AE estimates. In contrast to common practice, this study did not require that in order to be considered an AE, an event must prompt an intervention.10 As an example, the approach used in many studies would not include asymptomatic hypoglycaemia without an intervention as an AE, but instead consider it as a near miss. In Plint and colleagues’ study, 18% of AEs did not require an intervention, which could have led to higher estimates of AE rates.
Subjectivity of AE reviews and preventability assessments
Determining the presence of an AE generally has an inter-rater reliability with Cohen’s kappa ranging from 0.4 to 0.7, representing fair to good agreement.11 12 Issues that can affect determination of whether an event is considered an AE include the confidence standard used in making this decision, the types of AEs that a reviewer screens for (eg, negligent AEs vs all AEs) and the types of cases evaluated. Although retrospective record review remains a gold standard for AE detection, it is far from perfect. Reviewers are subject to hindsight and outcome biases. Events that look clear and linear in retrospect can be highly uncertain and probabilistic in real time. Medical decision-making is rarely documented in a way that provides retrospective insight and large parts of narratives related to events in the ED may be missing from medical documentation.
Determining AE preventability is likely to be even less objective, with kappas from 0.2 to 0.4, representing poor to fair agreement.11 12 Preventability determinations may be more subject to opinion, reviewer characteristics or even more capricious factors.
Factors that improve agreement in all AE reviews include having a smaller team of reviewers (preferable less than 5) and, to a lesser extent, greater reviewer experience and training.13 Clear written guidelines for reference during review meetings may also be helpful. Thus, both overall and preventable AE rates are highly dependent on both the review strategy and the threshold for calling an event an AE. Demonstration of only fair agreement in the determination as to whether an AE was preventable underscores how it is critical to have multiple rather than single reviewers in evaluating preventability. Future work should ensure multiple reviewers for preventability determinations and might include sensitivity analyses on AE rates using stricter and more liberal standards of AEs to probe the potential for variability.
Generalisability
In the USA, about 90% of children’s ED visits are to general EDs, with only 10% taking place in dedicated paediatric EDs such as those studied here.14 This is likely to be similar in other developed countries. Paediatric EDs have the highest readiness to care for children, with staff, supplies, resources, protocols and culture geared towards children’s emergency care.15 They also have the highest capability to care for children, indicating that they are by far the most likely to provide definitive emergency care and are therefore equipped with expertise across a range of both common and uncommon conditions.16 While general (ie, non-paediatric) EDs’ paediatric readiness has been increasing in the USA, hospitals’ capability to provide definitive paediatric care has been decreasing.16 This trend has been due in part to inpatient unit closures and has led to increased ED transfers of children, even for common, basic conditions such as asthma and gastroenteritis.17 18
Most EDs evaluate few children. Indeed, a third of US EDs evaluate fewer than five children per day.19 Much of the management of paediatric emergency conditions can be outsourced by transferring children to EDs with more paediatric experience, but this only occurs 5% of paediatric visits. What this means is that every ED owns most of its paediatric emergency care, whether experienced and resourced or not. How this translates to paediatric AE rates in general EDs is not known. AE rates may be lower (because of lower complexity paediatric patients), the same or higher (because of lower readiness and capability). As general hospitals decrease the provision of definitive paediatric care,16 it will be increasingly important to monitor the effectiveness and performance of emergency care systems that treat children. In particular, diagnostic safety has the potential to suffer from a decline in expertise, and attention to systematically measuring diagnostic safety in children is needed.
A final limitation related to generalisability is that care may differ between Canada and other countries. For example, Canadian EDs are known to be more parsimonious than US EDs in use of diagnostic tests for children, which could affect AE rates in either direction.20 Similarly, country-specific differences in expectations around follow-up and care coordination, the role of consultants during ED encounters and ED structure may also result in differences in AE rates.
Conclusion
With this study, Plint and coauthors have provided critical first steps towards a clearer understanding of the safety of paediatric emergency care. We believe key future steps include investigation of AE rates among children in general EDs is needed and study of the types of AEs that are sufficiently common and harmful that intervention is warranted. Given that most children do not visit paediatric EDs, evaluating quality and performance in all ED settings is a pressing concern, and is likely to become only more so in the future.
Ethics statements
Patient consent for publication
References
Footnotes
Funding KAM received funding through the Agency for Healthcare Research and Quality (award K08HS026503) and from the Boston Children’s Hospital Office of Faculty Development. RTG is supported in part by the Agency for Healthcare Research and Quality (grant 1 R01 HS027811-01) and by the Barnes Jewish Hospital Foundation (grants 3767 and 4442).
Disclaimer The contents of this work are solely the responsibility of the authors and do not necessarily represent the official view of the AHRQ, BCHOFD or the BJHF.
Competing interests None declared.
Provenance and peer review Commissioned; internally peer reviewed.