Background Adverse events (AEs) cause suffering for hospitalised children, a fragile patient group where the delivery of adequate timely care is of great importance.
Objective To report the incidence and characteristics of AEs, in paediatric inpatient care, as detected with the Global Trigger Tool (GTT), the Trigger Tool (TT) or the Harvard Medical Practice Study (HMPS) method.
Method MEDLINE, Embase, Web of Science and Google Scholar were searched from inception to June 2021, without language restrictions. Studies using manual record review were included if paediatric data were reported separately. We excluded studies reporting: AEs for a specific disease/diagnosis/treatment/procedure, or deceased patients; study protocols with no AE outcomes; conference abstracts, editorials and systematic reviews; clinical incident reports as the primary data source; and studies focusing on specific AEs only. Methodological risk of bias was assessed using a tool based on the Quality Assessment Tool for Diagnostic Accuracy Studies 2. Primary outcome was the percentage of admissions with ≥1 AEs. All statistical analyses were stratified by record review methodology (GTT/TT or HMPS) and by type of population. Meta-analyses, applying random-effects models, were carried out. The variability of the pooled estimates was characterised by 95% prediction intervals (PIs).
Results We included 32 studies from 44 publications, conducted in 15 countries totalling 33 873 paediatric admissions. The total number of AEs identified was 8577. The most common types of AEs were nosocomial infections (range, 6.8%–59.6%) for the general care population and pulmonary-related (10.5%–36.7%) for intensive care. The reported incidence rates were highly heterogeneous. The PIs for the primary outcome were 3.8%–53.8% and 6.9%–91.6% for GTT/TT studies (general and intensive care population). The equivalent PI was 0.3%–33.7% for HMPS studies (general care). The PIs for preventable AEs were 7.4%–96.2% and 4.5%–98.9% for GTT/TT studies (general and intensive care population) and 10.4%–91.8% for HMPS studies (general care). The quality assessment indicated several methodological concerns regarding the included studies.
Conclusion The reported incidence of AEs is highly variable in paediatric inpatient care research, and it is not possible to estimate a reliable single rate. Poor reporting standards and methodological differences hinder the comparison of study results.
- Adverse events, epidemiology and detection
- Chart review methodologies
- Trigger tools
Data availability statement
Data are available in a public, open access repository. Data and R code are available on 10.5281/zenodo.7335611.
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
WHAT IS ALREADY KNOWN ON THIS TOPIC
The only available systematic review in this area is dated and shows a surprisingly low estimate of adverse event (AE) incidence. As paediatric inpatients are particularly vulnerable and run a high risk of exposure to AEs, a systematic review examining this important knowledge gap is lacking.
WHAT THIS STUDY ADDS
This review gives an up-to-date estimate of the incidence and variation of paediatric inpatient AEs. It also adds relevant methodological reflections about structured retrospective record review methods, as well as their application and reporting quality.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
A better knowledge of the complex nature of paediatric AEs is important for the development of more targeted patient safety interventions to increase quality of care and prevent paediatric patients suffering AEs. An awareness of the current incomplete reporting of key elements related to AE data may help researchers to improve the quality of reporting in future studies.
Adverse events (AEs) are costly,1 cause suffering for patients, their families and for healthcare professionals2 and have been recognised as a critical global healthcare issue.3 4 An AE may be defined as unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalisation, or that results in death.5 The incidence of AEs varies between contexts (eg, country, hospital types, included specialities) and research is heavily influenced by the method used. Between 7% and 40% adult general inpatients are affected by AEs. These are often deemed to be preventable, indicating that patient safety can be improved.6
Hospitalised children are a fragile patient group. Even a low degree of error related to acts of omissions or commissions can affect the child’s health and in the long-term risk affecting the child’s development and future.7 Patients treated at intensive care units run a greater risk of being exposed to AEs than general care patients.8 9 Sedation and the need for intravascular and/or breathing devices are factors associated with AEs in paediatric patients. Those patients experiencing AEs are on average younger and have a longer length of stay.8
There are various methods for detecting, measuring and characterising AEs in healthcare, but as yet no gold standard exists.10 A commonly used method is structured retrospective record review, which includes different approaches, for example, the Harvard Medical Practice Study (HMPS) method11 12 or the Global Trigger Tool (GTT)5 with its subsequent adaptations (Trigger Tools, TTs) to be used in different contexts, such as paediatrics,13 14 oncology,15 psychiatry16 or home healthcare.17 Record review has been shown to be superior in detecting AEs compared with other methods, such as incident reporting systems and administrative data.14 18–20
In adult care, several systematic reviews6 21–24 regarding the identification of AEs using record review methodology, with or without meta-analysis, have been published. To the best of our knowledge, only one systematic review focusing on paediatric care has been published.25 This review included nine publications, of which six used record review data and three used administrative record data, and was restricted to a minimum of 1000 patients. The admission year for included patients ranged from 1984 to 2009. This review presents a surprisingly low AE incidence. The publications of GTT and TT studies in the paediatric context have increased in the last 10 years. Therefore, an updated systematic review, irrespective of study sample sizes, was indicated. The aim of this systematic review is to report the incidence and characteristics of AEs, in paediatric inpatient care, as detected with the GTT, the TT or the HMPS method.
The review was carried out as a systematic review and meta-analysis. The study protocol was uploaded on h 10.5281/zenodo.5513354.
Information sources and search strategies
The following databases were used for the search: MEDLINE, Embase, Web of Science and Google Scholar.26 A search strategy was developed with the help of librarians, and this encompassed subject headings and free text words that described the population, the context, the concept and type of evidence source. The search terms used were: Iatrogenic Disease, Medical Errors, Patient Harm, adverse event*, harm, trigger*, Adolescent, Child, Infant, p?ediatric*, neonat*, child*, newborn* infant*, adolescen*, premature*, preschool, teenager*, Hospitals, Inpatients, Hospitalisation, Hospital Units, Hospital Departments, hospital*, intensive care, inpatient*, review*, record*, chart*, trigger tool and Harvard Medical Practice*. The systematic searches were performed between 4 and 8 June 2021 and no restrictions in language or publication year were applied. The full search strategy and outcomes for the respective database are shown in online supplemental material 1, tables S1–S4. Furthermore, the search was supplemented in the data extraction process with a manual scan of the reference lists of eligible publications.
Publications that met the following criteria were included: (1) Children, all age groups, if cared for in paediatric inpatient units; (2) Studies including both adults and paediatric patients if the data for paediatric patients were reported separately; (3) Peer reviewed full text primary publications, reporting relevant quantitative outcome data; (4) Studies applying manual retrospective medical record review using GTT, TT or HMPS methodologies. We accepted all types of AE definitions (online supplemental material 1, table S5).
The following exclusion criteria were applied: (1) Publications reporting AEs for paediatric patients with a specific disease/diagnosis/treatment/procedure or who were deceased; (2) Studies in primary care, psychiatric care, day care/ambulatory care and emergency departments or other outpatient units at the hospital; (3) Study protocols without AE outcome data; (4) Publications such as conference abstracts, editorials and systematic reviews; (5) Studies using, for example, clinical incident reporting systems as the primary data source where these incident reports were subsequently analysed using record review; and (6) Publications reporting only specific AEs, for example, adverse drug events (online supplemental material 1, table S5).
The first screening step of applying the eligibility criteria to titles and abstracts was done independently by four reviewers, working in pairs (MU/PD, UF/LB). Thereafter, eligible full texts were retrieved, and the same reviewers independently assessed full texts. The reason for exclusion was noted and any discrepancies between the individual reviewers were discussed in the pairs until consensus was reached. If required, discussion was held with the whole research group. Discussions during the selection process mostly concerned whether multiple publications on the same study were considered as overlapping or not.
Data extraction process
To ensure quality, data were independently extracted by two researchers per publication. Data regarding key study characteristics (eg, sample size, setting, number of hospitals, method used, patient demographics) and patient outcomes (incidence, frequencies, preventability, types, severity) were collected. Authors of 27 primary studies were contacted by email to request additional information to calculate the primary outcome or part of the secondary outcomes. Information was provided from 17 studies. Any discrepancies between reviewers were resolved in the same way as in the selection process and a consensus for each study was reached. All the studies included were discussed at some point within the research group. Discussions were either related to the quality assessment, the methodology or interpretation of data.
To assess the methodological quality of each included study, a previously used quality assessment tool (QAT) was adapted. This QAT was based on the structure of the Quality Assessment Tool for Diagnostic Accuracy Studies 2 tool27 and the content of the QAT by Musy et al28 and later by Eggenschwiler et al29 (online supplemental material 1, page 14). The QAT consists of five domains: patient selection, reviewers, record review process, outcomes and flow. Each domain includes two to three signalling questions which form the basis for the assessment of risk of bias and applicability-related concerns. These were rated as either low, high or unclear. Expert knowledge in quality assessments and record review methodology guided the adaptations. Examples of adaptations used were revisions of the domain record review process with signalling questions regarding support and monitoring during the review process. Furthermore, the risk of bias and applicability-related concerns were also rated as an overall judgement for each study (online supplemental material 2). The QAT for each study was used by two reviewers independently and a consensus was reached.
A meta-analysis was carried out with the percentage of admissions with ≥1 AEs as the primary outcome measure.
Secondary outcomes were AEs per 100 admissions, AEs per 1000 inpatient days, percentage of preventable AEs, as well as percentage of admissions with preventable AEs. In addition, types of AEs and AE severity were described.
Analyses were conducted using R V.4.1.3 on Linux30 with the meta31 and metafor32 packages. All statistical analyses were stratified, distinguishing general and intensive care populations, as these are known from the literature to differ in the distribution of AEs.6 22 33 They were also stratified by the record review methodology used (GTT/TT or HMPS). The categorisation of the two populations was based on whether most patients were admitted to either general or intensive care units. Studies using the HMPS methodology did not predominantly include intensive care patients. The GTT and TT methodologies were analysed together, as these methods share the same conceptual approach.
Where not explicitly reported, we calculated the number of admissions with ≥1 AE from the reported percentage estimates of admissions with AEs. Similarly, we derived the number of patient days by dividing the total number of AEs by the reported rate of AEs per 1000 patient days. Studies using the HMPS methodology were excluded from the meta-analyses for AEs per 100 admissions and AEs per 1000 patient days. Most of these studies included only the most severe AE per admission and therefore the estimates were not comparable.
We fitted random intercept logistic models, using the R metaprop function with the Wilson method for CIs for the meta-analysis of the percentage of admissions with ≥1 AE, the percentage of preventable AEs and the percentage of admissions with preventable AEs.31 For the AEs per 100 admissions and AEs per 1000 patient days we used random intercept Poisson models, fitted with the R metarate function.32
Other systematic reviews on the same topic reported I2 values of up to 100%20 21 23. Although frequently reported I2 is not valid in the context of single proportions. We decided to characterise the variability of the estimates by reporting prediction intervals (PIs).34 35 The 95% PI quantifies the sample variability and is expected to capture estimates from future studies with a 95% level of confidence.36 We identified high heterogeneity, illustrated by the width of the PIs, which is wider than the 95% CI in the presence of between-study heterogeneity. Hence, we focused our reporting on PIs rather than CIs. Furthermore, we investigated heterogeneity via stratified analyses of five elements relating to risk of bias and four connected to applicability-related concerns. P values, derived from the likelihood ratio test for model fit, were considered statistically significant with a value of p<0.05. The PRISMA 2020 guideline for reporting systematic reviews was applied.37
The database searches yielded 3790 publications of which 1317 were duplicates leaving 2473 unique publications which were screened by title and abstract. In total, 108 publications underwent full text screening, including four publications from reference lists. After assessment of eligibility, 64 publications were excluded and 44 publications8 11–14 33 38–75 of 32 unique studies8 11 12 14 33 38–42 45 48–50 53 55 57–68 71–74 were included (online supplemental material 1, figure S1). As one study55 reported outcomes for both populations, a total of 33 samples were included, 22 for the general care and 11 for intensive care populations.
The studies were published between 1991 and 2021 with inclusion periods ranging from year 1984 to 2019 and 59.4% of the studies were published in the last 10 years. The study periods ranged from 1 month to 6 years. The 32 studies originated from 15 countries, of which 34.4% were from North America, 28.1% from Europe, 18.8% from South America, 9.4% from Australia, 6.3% from Africa and 3.1% from Asia. In total, 33 873 paediatric admissions (median, 330; range: 11–6661) and 124 800 patient days (median, 2743; range, 87–21 789) were included. A wide variation of units was found, and 68.8% (n=22) of the 33 samples included mainly general care (eg, surgical, medical) and 34.4% (n=11) included mainly intensive care units for paediatric and neonatal patients. Patients’ mean age (n=14 studies) varied between 3.0 years to 7.8 years and mean length of stay (n=17 studies) 2.8 days to 22.8 days (table 1). Most of the studies (n=28) were written in English, three in Spanish and one in Portuguese.
Study methodology characteristics
A majority used GTT/TT (n=23, 71.9 %), followed by HMPS (n=9, 28.1%). No study published after 2014 used the HPMS method. The most frequent sampling strategy was random (n=26, 81.3%). A majority of the 32 studies (n=25, 78.1%) were assessed to have used a two-stage retrospective record review process and the number of triggers/screening criteria varied between 14 and 88. Twenty-six (81.3%) described training prior to the review process (table 2) and 12 studies used test records.
Seven studies (21.9%) had teams where the whole or part of the team had prior experience in record review methodology and seven studies (21.9%) reported support during the review process, such as expert consultation. Ten studies (31.3%) described a monitoring process to ensure completeness, consistency and accuracy (data not shown).
Both acts of omissions and commissions were included in 53.1% (n=17) of the studies and 78.1% (n=25) included ≥1 AE per patient. Outcomes for inter-rater reliability, using double reviews, were reported in 53.1% (n=17). Of those, kappa values were reported in five (26.3%) studies, percentage agreement in four (21.1%), and both measures in eight (42.1%). Half of the studies included AE(s) that occurred both before, during and after index admission, and eight studies (25.0%) didn’t specify the time frame for inclusion. The GTT manual’s AE definition or similar was used in 17 studies (53.1%) and the HMPS definition or similar in 10 (31.3%) (table 2), and 77.8% (n=25) of these had a reference to their AE definition. Preventability was assessed in 19 studies (59.4%) (table 2).
The total number of identified AEs was 8577 (range 0–34 per patient) in 33 samples, 3459 (range 0–27 per patient) in the general care population (13 GTT/TT and 9 HMPS samples) and 5118 (range 0–34 per patient) in the intensive care population (11 samples). Preventability was reported in 16 samples (48.5%) with a total of 3785 identified preventable AEs (online supplemental material 1, table S6).
The most common types of AEs in general care (n=9 studies) were nosocomial infections (range, 6.8%–59.6%), medication-related (2.3%–48.6%) and surgical-related (0.9%–30.5%). Pulmonary-related (10.5%–36.7%), nosocomial infections (6.6%–40%) and medical technical product-related (1.3%–30.8%) were the most common types of AEs in intensive care (n=8 studies) (table 3).
Twenty-one studies assessed and described the severity of paediatric AEs. A majority of these (71.4%, n=15) used a modified version of the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Scale (online supplemental material 1, table S6). The studies assessing severity by the modified NCC MERP Scale, irrespective of population, had a range for minor consequences, category E, between 16.5% and 88.4% (mean, 56.9%); major, category F, 0.0% and 62.7% (28.9%); permanent, category G, 0.0% and 14.8% (4.0%); life-threatening, category H, 0.0% and 28.9% (7.4%) and death, category I, 0.0% and 15.7% (2.7%). The intensive care population had a mean of 11.7% for the two most severe categories—life-threatening and death, whereas the general care population had a mean of 3.1%.
The forest plot in figure 1 shows the primary outcome, that is, percentage of admissions with ≥1 AEs for 32 out of 33 samples. The range of percentage of admissions ≥1 AEs for GTT/TT was 6.1%–38.0% and 16.2%–83.9% for general and intensive care and the equivalent for HMPS was 0.0%–19.0%. The pooled estimates for the GTT/TT (general and intensive care populations) were 17.7% (95% PI 3.8%–53.8%) and 47.3% (95% PI 6.9%–91.6%), respectively, and 3.9% (95% PI 0.3%–33.7%) for the HMPS (general care). There was a statistically significant difference in the pooled estimates between the two populations within the GTT/TT methodology (p=0.0003).
Online supplemental material 1, figures S2–S5 present forest plots for the secondary outcomes. In 24 samples (GTT/TT), AEs per 100 admissions (general care, range 6.8–93.8; intensive care, 30.2–325.0) were supplied or could be calculated. The pooled estimate for the general care population was 24.8 AEs per 100 admissions (95% PI 4.2–145.2) and 103.6 AEs per 100 admissions (95% PI 5.3–699.7) for intensive care (table 4; online supplemental material 1, figure S2). An overview of the pooled estimates and related measures for the primary and secondary outcomes is shown in table 4.
In 22 samples (GTT/TT), AEs per 1000 patient days varied between 15.5 and 390.8 for general care and 22.6 and 599.1 for intensive care. The pooled estimates for AEs per 1000 patient days were 48.3 (95% PI 5.9–393.1) and 126.2 (95% PI 6.4–2495.1) for general care and intensive care, respectively. Half of the studies for intensive care had over 100 AEs per 1000 patient days (range 195.7–599.1) (table 4; online supplemental material, figure S3).
Of the 16 samples that reported preventability, the pooled percentage of preventable AEs for GTT/TT (general and intensive care populations) was 58.6% (95% PI 7.4%–96.2%) and 67.4% (95% PI 4.5%–98.9%). The corresponding for the HMPS was 53.2% (95% PI 10.4%–91.8%) (table 4; online supplemental material, figure S4). The pooled percentage of admissions with preventable AEs (12 samples) was for the GTT/TT (general and intensive care) 7.3% (95% PI 0.0%–100.0%) and 25.0% (95% PI 2.5%–81.3%) and for HMPS 2.3% (95% PI 0.0%–59.3%) (table 4; online supplemental material, figure S5).
Quality assessment and sensitivity analysis
Several methodological concerns were identified during the quality assessment process.
Concerning overall assessments, risk of bias was assessed as high in 85% compared with 44% in the 9 GTT/TT and 13 HMPS studies for the general population and 100% for the intensive care population (n=10, GTT/TT). When compared with GTT/TT studies, HMPS studies more frequently had both a low risk of bias with low applicability concerns at the domain level (online supplemental material 1, table S7, figures S6-S7).
The stratified analysis exploring heterogeneity was based on the quality assessment and percentage of admissions with ≥1 AE as the outcome. Lower AE outcomes were detected where the risk of bias was rated as high or unclear in the domain ‘record review process’ than in those with a low risk of bias for general care (GTT/TT) (online supplemental material 1, figure S8). For the HMPS methodology, variation is driven by the unclear category, which hampers interpretation (online supplemental material 1, figure S9). For the intensive care population, studies with high risk of bias detected lower levels of AEs in the domain ‘patient selection’ than those rated as low risk of bias (online supplemental material 1, figure S10). In all three strata, high risk of bias for the domain ‘outcomes’ was typically associated with higher AE rates compared with low risk of bias. Nevertheless, the limited sample size does not provide enough evidence to draw any solid conclusions.
We conducted a systematic review and a meta-analysis, consisting of 32 studies with 44 publications examining the incidence and characteristics of AEs detected using three commonly used record review methods (GTT, TT and HMPS). Nosocomial infections were common in both populations and most of the AEs were less severe. There was substantial between-study heterogeneity and overall high risk of bias in most studies. The PIs for the primary outcome for GTT/TT studies were 3.8%–53.8% and 6.9%–91.6% (general care and intensive care populations) and 0.3%–33.7% for the HMPS studies (general care). The PIs for the percentage of preventable AEs for GTT/TT studies were 7.4%–96.2% and 4.5%–98.9% (general care and intensive care) and the equivalent for HMPS studies was 10.4%–91.8%.
Incidence and characteristics of adverse events in paediatric inpatient care
Our review confirms substantial heterogeneity between general care and intensive care studies, as well as between methodologies (GTT/TT and HMPS). However, the results also display a high level of heterogeneity within populations. The degree of heterogeneity is in accordance with previously published systematic reviews.22 23 25 76 The majority of studies were judged to be at high risk of bias, which also lowers the trust we place in the summary estimates. Therefore, caution is needed when drawing conclusions from the pooled data of the combined studies. We urge the reader to focus on the given PIs when interpreting the pooled data.
Berchialla et al25 focused solely on paediatric inpatient AEs in their systematic review and reported a pooled incidence of AEs at 2.0%. This is lower than the pooled incidence of admissions with ≥1 AE using the GTT/TT methodology, shown in the present review: 17.7% for the general care population and 47.3% for the intensive care population. However, it is in line with the 3.9% for studies conducted using HMPS methodology. Their inclusion of studies using only the HMPS’s AE definition may partly explain the difference, as the threshold for inclusion of an AE is higher due to the requirement of temporary or permanent disability, death or increased length stay. This could have led to minor, but perhaps commonly occurring AEs, being excluded with the risk of underestimation of AEs as a consequence.
Sauro et al23 included, in addition to record review, other data collection methods, and found a pooled estimate of 1.4 AEs per 100 paediatric admissions and up to 11.9 AEs per 100 admissions in adult care. This is also considerably lower than our corresponding estimate for GTT/TT studies at 24.8 and 103.6 (general care and intensive care populations) AEs per 100 admissions. On the other hand, a newly published systematic review29 including GTT/TT studies in general care reported a pooled incidence of 30.0 AEs per 100 adult admissions which is higher compared with our findings in the general care population.
Half of the studies assessed and reported preventability, both for general and intensive care, and around 60% of AEs were identified as preventable. However, as discussed by Hibbert et al,6 the assessment of preventability is a subjective judgement and comparison between studies is to be done with caution. This, therefore, is a methodological limitation. Panagioti et al22 included both adult and paediatric populations and showed an overall pooled prevalence of 6% for preventable AEs. This is in line with our preventability estimates in GTT/TT studies of 7.3% for admissions in general care but is higher compared with the 2.3% found in the HMPS studies. However, we found a pooled estimate of 25% for preventable AEs for intensive care compared with 18% in the study by Panagioti et al.22
A longitudinal retrospective record review study indicated an increased frequency of AEs over time, where one explanation was the increased number of patients with less complex conditions receiving day and outpatient care instead of inpatient care. This leads to an increased proportion of seriously ill patients in hospitals, and this may affect the AE rates for inpatient care.77
Important aspects of the variation in AE rates are the context and case mix of patients such as inclusion of units, medical specialities, hospital types, academic level of the hospital, patient age and comorbidity, and level of care. In both general care and intensive care populations, nosocomial infection was among the most common type of adverse event, also identified as one of the main causes of morbidity and mortality for paediatric inpatients.78 Paediatric patients have many risk factors for infections related to, among other things, immunodeficiencies and poor skin barrier. Skin harm is a predisposing factor for nosocomial infections,79 and was the overall third most common type of AEs in the current review. It is important to keep in mind the considerable variation regarding the taxonomy of reported types of AE used, which makes comparisons between studies difficult.
The use of record review methodology for specific populations seems to have increased over the last few decades. All studies conducted solely in the intensive care population were conducted after 2006 and a vast majority in the last 10 years.
We could not sufficiently explain the heterogeneity in the primary outcome using the quality of the studies. Insufficient reporting affected the risk of bias and applicability-related concerns negatively. The high risk of bias for the domain ‘outcomes’ was typically associated with a higher percentage of admissions with an AE. Sauro et al23 reported, in accordance with our findings, a significantly higher pooled estimate of AEs for lower-quality studies. Furthermore, they showed, in consistency with the current study, that the presence of AEs at admission was unclear.
Many methodological limitations and reasons for the variations of AE outcomes in published studies have been suggested, for example, patient record documentation, the experience of the review team, quality assurance activities, inclusion criteria, AE definitions, choice of triggers and time frame for inclusion of AEs.6 Apart from the researchers’ adaptations, some variations may be explained by the different record review methods. Although, it would have been very interesting to analyse the variation based on the different methodological applications, it was outside of the scope of this review. In a recently published meta-analysis for adult inpatients, some of the variation could be explained by those methodological aspects (type of hospital included, age of sample included and experience of the review team).29
Another aspect is that variables that might affect the estimates of AE outcomes were not always clearly specified in the studies, for example, the time frames for AE inclusion. As a consequence, data extractors made interpretations based on triggers, for example, hospital readmission within 30 days. Another example is the inclusion of acts of commissions and/or omissions which was often not explicitly specified in the studies. GTT and TT studies following the Institute for Healthcare Improvement’s manual exclude AEs related to acts of omission which could lead to an underestimation of AEs. Wilson et al12 found in their study that acts of omission were nearly twice as common as acts of commission. Hibbert et al6 suggest that several additional variables should be included when using GTT, for example, omissions, preventability and other characterisations, to get a better understanding of AEs. This suggestion is in accordance with the HMPS methodology, where AEs are categorised to a higher extent compared with GTT. To summarise, as many studies use minor adaptations of the record review process,19 80 the reporting of AEs would benefit from a standardised guideline. This would decrease the methodological heterogeneity, thereby increasing replicability, interpretations and comparisons.
Despite variations between inpatient care, AE outcomes and measurements, the high incidence of AEs and percentage of preventable AEs indicate that there is more to be done regarding patient safety interventions. Zegers et al81 made an umbrella review concerning evidence-based interventions to reduce inpatient AEs and they conclude a need for more high-quality studies to determine what interventions will have the most positive impact on patient safety. However, they state that there is evidence available for interventions to prevent infections, falls, delirium, adverse drug events, cardiopulmonary arrest and mortality. Furthermore, the measurement of AEs must be incorporated as part of the learning system within healthcare organisations and be connected to evidence-based interventions and evaluation of these as part of the continuous improvement work as measurement alone does not create safe care.82
Strengths and limitations
The adoption of a robust search strategy using several databases with no limitations in publication dates or language of publication lessens the likelihood that important studies were missed and may have changed the estimates in a significant way. However, the possibility of missing potentially relevant studies meeting the inclusion criteria is always present as we did not search for ‘grey’ literature. We did not use funnel plots to explore publication bias or other biases associated with small study size, as patterns of publication bias in the field of single-arm studies reporting proportions is not well understood and also because funnel plot analyses can lead to inaccurate conclusions.83A rigorous approach was adopted to the screening and data extraction process, as well as the assessment of bias and applicability. The large number of studies included further strengthens the study. We also contacted the authors for several of the studies where vital variables were missing. This led to fewer variables being categorised as not specified and therefore fewer studies were excluded from the meta-analyses.
One limitation is that the exclusion criteria disqualified studies with, for example, only automated AE detection, those including only outpatients or studies focusing on a specific diagnosis, treatment, or AE such as adverse drug events. This could have reduced the number of eligible studies and the final sample size as estimates could differ from estimates in a wider population. Concerning generalisability, most studies were conducted in Europe, as well as North and South America. Last but not the least, critically ill patients need complex care, which puts them at risk for AEs.3 As previously stated, paediatric patients run a high risk for AEs during inpatient care, in general care, but specifically in intensive care.7 Some of the heterogeneity within the general care population might be explained by the fact that several studies in the general care population also included intensive care patients to some extent. We choose to include a heterogeneous group of studies to provide estimates of paediatric inpatient AEs to represent the diversity of hospital settings, as well as to include the three most common record review methodologies.
For the reporting of the meta-analysis, we have taken the decision to not report on I2 values. This measure can be used to compare statistical heterogeneity but not clinical heterogeneity.84 Rücker et al84 recommends using τ2 to assess clinical heterogeneity. IntHout et al34 go a step further and recommend presenting PIs instead, as it is presented on the same scale as the outcome measure in contrast to τ2 or I2. Therefore, we opted to provide PIs as measures of heterogeneity.
We acknowledge a deviation from the published study protocol, as we changed our primary outcome measure during the data-extraction phase, before conducting any statistical analyses. The percentage of admissions with ≥1 AE was chosen instead of AEs per 100 admissions, because this was the only measure with which we could directly compare the two methodological groups of GTT/TT and HMPS.
This review demonstrates a large between-study variation in estimates of the incidence of paediatric AEs. It also highlights the importance of a thorough understanding of the complex nature of AEs, and the sources of variation and of bias. The current lack of reporting standards in this field impedes comparison of study results. To advance the field of record review methodology, new reporting and risk of bias guidance tools are needed to enhance both comparability and overall quality of the studies and to maximise impact of study findings.
Data availability statement
Data are available in a public, open access repository. Data and R code are available on 10.5281/zenodo.7335611.
Patient consent for publication
The authors wish to thank the librarians Alena Lindfors, Ulrika Gabrielsson and Kristina Lönn at Dalarna University for support with the development and tests of search strategies as well as the performance of the database searches.
The authors would also like to thank Paula Kelly-Pettersson for the editing of this manuscript. An acknowledgement also to the ones who provided supplemental data for the data analyses: Swati Agarwal, Ross Baker, Laura Barrionuevo, Troyen Brennan, Peter Davis, Sana El Mhamdi, María Eugenia Esandi, Virginia Flintoft, Robert Giddered, Stephen Hancock, Facundo Jorro Barón, Alexander Kiss, Gitte Larsen, Roy Lay-Yee, Lucian Leape, Mondher Letaief, Anne Matlow, Geetanjali Shankarprasad Salimath, Stephan Schug, Paul Sharek, Anne Lee Solevåg, Michael Soop, Carin Verlaat and Jentien Vermeulen.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
PD and LCE are joint first authors.
Twitter @l_eggens, @AnneRutjes, @msimoninfo
Correction notice This article has been corrected since it was first published. The open access licence has been updated to CC BY.
Contributors PD, LCE, AWSR, LB, SNM, MS, UF and MU contributed to conceptualisation of the project and to methodology. PD, LCE and MU carried out data curation. LCE performed formal analysis with support from AWSR and GM. PD, LCE, UF and MU wrote the original draft. All authors contributed to review and editing of the manuscript. PD and LCE contributed equally to this paper. Author responsible for the overall content is MU.
Funding This study was funded by grants from a regional agreement on clinical research (ALF) between Region Stockholm and Karolinska Institutet (2020-0443), Childhood Foundation of the Swedish Order of Freemasons (no award/grant number).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.