You are here

  • Home
  • Archive
  • Volume 4, Issue 7
  • Prevalence and severity of patient harm in a sample of UK-hospitalised children detected by the Paediatric Trigger Tool

Article Text

Article menu
XML
Paediatrics
Research
Prevalence and severity of patient harm in a sample of UK-hospitalised children detected by the Paediatric Trigger Tool
  1. Susan M Chapman1,2,
  2. John Fitzsimons3,4,
  3. Nicola Davey5,
  4. Peter Lachman1
  1. 1Department of Paediatrics, Great Ormond Street Hospital
    for Children NHS Foundation Trust, London, UK
  2. 2University College London, London, UK
  3. 3Department of Paediatrics, Our Lady of Lourdes Hospital, Drogheda, Ireland
  4. 4Quality & Patient Safety Division, Health Service Executive, Dublin, Ireland
  5. 5NHS Institute for Innovation and Improvement, University of Warwick Science Park, Coventry, UK
  1. Correspondence to Dr Peter Lachman; peterlachman{at}mac.com

Abstract

The measurement and examination of adverse events (AEs) that occur in children during hospital admissions is essential if we are to prevent, reduce or ameliorate the harm experienced. The UK Paediatric Trigger Tool (UKPTT) is a method of retrospective case note review that measures harm in hospitalised children.

Objectives To examine the harm resulting from the processes of healthcare in hospitalised children from centres providing data to the National Health Service (NHS) Institute UKPTT data portal, to understand the positive predictive values of triggers and to make recommendations for the further development of the trigger tool.

Setting 25 hospitals across the UK, including secondary, tertiary and quaternary paediatric centres.

Participants Randomly selected children who were admitted to hospital for longer than 24 h.

Outcome measures The primary outcome measure was the rate of harm (the percentage of children experiencing one or more AEs during a hospital admission). Secondary measures were the severity of harm and performance of triggers.

Results Data from 3992 patient admissions were reviewed across the hospitals and submitted to the trigger tool portal from February 2008 to November 2011. At least one AE was reported for 567 (14.2%) patients, with 211 (5.3%) experiencing more than one event. There were 1001 AEs identified. Where harm occurred, it was considered temporary for 923 (92.2%) AEs; however, 43 (4.3%) AEs resulted in the need for life-sustaining interventions, 18 (1.8%) AEs led to permanent harm and for 17 children (1.7% of AEs) the AE was believed to have contributed to death.

Conclusions There is a significant, measurable level of harm experienced by children admitted to hospitals in the UK. While most of this harm is temporary, some of it is serious. The UKPTT offers organisations the means to measure and examine the AEs occurring in their hospital in order to reduce harm.

  • Paediatrics

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

https://doi.org/10.1136/bmjopen-2014-005066

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This study estimates that one in seven children experience harm during admission to hospital in the UK. Most of this harm is temporary, but a significant minority is serious. This should generate discussion about patient safety in paediatrics.

    • The study used the UK Paediatric Trigger Tool that can be used by any hospital to learn about harm in order to prevent or reduce it.

    • The study does not distinguish between preventable and non-preventable harm; however, we believe that there is an opportunity to learn from any harm event.

    Introduction

    The provision of care that is safe and reliable is a fundamental goal of modern healthcare. Patient safety is the prevention, reduction and amelioration of medical harm.1 ,2 Medical harm (synonymous with the terms patient harm and adverse event, AE) is defined as unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalisation or that results in death.3 Efforts to improve patient safety have been hampered by a lack of reliable data on the prevalence and nature of harm in all areas of practice. Patients and healthcare professionals need to understand the burden of harm in healthcare in order to develop effective interventions.4

    Development of Trigger Tool methodology

    The Global Trigger Tool (GTT) for measuring harm was developed by the Institute for Healthcare Improvement for use in adult care settings.3 Trigger tools have also been developed for specific populations and settings, including acute hospitals, surgery,5 critical care6 and primary care.7 One study used the GTT to measure harm at a large academic children's hospital in the USA and recommended the development of a paediatric specific tool.8 Paediatric-specific trigger tools have been developed for neonatology,9 paediatric critical care,10 ,11 medications12 ,13 and a general trigger tool for harm in hospitalised children.14 A UK version of the acute adult GTT had already been developed, but this was not applicable to a paediatric population. In 2008, the National Health Service (NHS) Institute for Innovation and Improvement undertook to develop a UK Paediatric Trigger Tool (UKPTT) that could be applied to all levels of acute paediatric care.

    Trigger tool method for reviewing case notes

    The trigger tool method is a retrospective review of 20 sets of healthcare records each month, using a standardised methodology. A random sample of 20 inpatient case notes is selected using a randomisation matrix on a monthly basis. The healthcare record is examined in a structured process for 20 min to search for ‘triggers’. A ‘trigger’ is a predefined event that alerts the reviewer to the possibility of patient harm. Once a trigger is identified, the reviewer uses clinical expertise to examine the records in more detail to understand the circumstances around the event. If harm is suspected, a second reviewer (usually a physician) is consulted to confirm and grade the AE using the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) grading system (table 4).15

    An example of a trigger is the administration of the antidote medication naloxone (a trigger) to reverse the effects of opiates. This alerts the reviewer to a possible overdose of opioids. The reviewer examines the relevant parts of the healthcare record to assess whether the use of naloxone was for this reason or not. If this is the reason, then the harm is graded (see online supplementary file).

    Development of the UKPTT

    In 2008, the NHS Institute sponsored the development of a Paediatric Trigger Tool because at the time there was no tool available for hospitalised children. The tool design was informed by the early (and prepublication) findings of a Canadian Paediatric Trigger Tool (CPTT) study,14 and the UK Acute Trigger Tool for adults.16 The development was a coproduction involving the collaboration of patient safety experts from the NHS Institute, international leaders in Paediatric Trigger Tool development, and clinical experts from nine UK hospitals including children's hospitals and district general hospitals. Following discussion and testing, the group agreed on 40 paediatric oriented triggers to be included in the UKPTT. These were based on the triggers used in other tools, UK evidence of AEs and the experience of the reference group in harm and AEs (a subset of the coproduction group). Production of a UK tool was intended to enhance ownership by the clinicians, who would use it in practice and to modify triggers that were not appropriate for the UK setting. We also added a category for ‘other harm’ to capture harm that was not detected by one of the listed triggers.

    The UKPTT advocates a working definition of patient harm as ‘anything, which you would not like to happen to yourself or a member of your family as a result of, or contributed to by, medical care’. The decision to aim for a broad definition was to focus on the patient rather than on the medical system—a less defensive approach. This is a broader definition than that given by Griffin and Resar3 or the Canadian tool and aimed to encourage clinicians to explore a holistic concept of harm than that traditionally reported. It allows the inclusion of acts of omission as well as commission. The definition includes missed or delayed diagnosis along with physical and psychological harm.

    Through the coproduction, support was developed for UKPTT users such as face-to-face training, online and printed guidance and standardised data collection forms.

    Data collection

    As part of the UKPTT development, the NHS Institute created a web-based trigger tool portal into which participating hospitals entered anonymised data. The portal calculated harm rates and produced run charts that hospitals could download. Contributing hospitals consented to their data being collated and published to further the understanding of harm in hospitalised children in the UK. Participating hospitals developed local administrative and governance arrangements for PTT reviews following the standard guidance (see online supplementary file).

    Aims and methodology

    The aims of this study are to:

    1. Describe the rate and severity of harm occurring in hospitalised children from UK centres submitting data to the NHS Institute's Trigger Tool portal.

    2. Report the frequency and positive predictive value (PPV) of triggers to detect harm.

    3. Make recommendations for further application and development of the tool.

    Participating hospitals, which voluntarily decided to use the PTT, included secondary, tertiary and quaternary centres. Reviewers were trained in trigger tool methodology either by experts at the NHS Institute or by using online resources with telephone support. Data were collected through the online trigger tool portal that opened in February 2008.

    Results

    Data from 3992 case note reviews from 25 hospitals submitted to the trigger tool portal between February 2008 and November 2011 were analysed. Nine of the hospitals were children's hospitals; the remainder was classed as district general hospitals. Data from four additional hospitals that used the portal were excluded because they each submitted less than 10 case entries. Harm was recorded as occurring for 567 patients (14.2%) while the majority (85.8%) of patients experienced no evidence of harm. Reviewers identified 1001 AEs, an average of 1.8 events per patient experiencing harm.

    There was considerable variation between hospitals in the number of case notes reviewed (12–622), the number of triggers detected (17–1877) and the overall harm rate reported (0—73.3%). Results from each hospital are reported in table 1. Of the 567 children who suffered an AE, the majority (n=356, 63%) experienced a single event. However, 211 (37%) patients suffered more than one AE within the same admission. One patient was reported to have suffered 10 AEs in a single admission. A summary of the number of AEs per case is presented in table 2.

    View this table:
    Table 1

    Number of case reviews, positive triggers and adverse events by individual hospital

    View this table:
    Table 2

    Number of AEs per patient

    Individual triggers varied in their ability to lead to detection of harm. The trigger Complications of procedure or treatment yielded the greatest amount of harm (182 AEs). The PPV varied from 80.0% for surgical site infection to 2.62% for missing observations/early warning scores. Also, the PPV was generally low in frequently identified triggers, such as missing observations/early warning scores (PPV 2.6%) and unplanned admission (PPV 4%). The positive triggers, AEs and PPV for each trigger are displayed in table 3.

    View this table:
    Table 3

    Trigger descriptors, AE and positive predictive value

    The majority of AEs (n=923, 92.2%) resulted in temporary harm to the patient (grades E and F—see table 515); 43 AEs required life-sustaining interventions and 18 resulted in permanent harm. In 17 cases, the AE was believed to have contributed to the child's death (table 4).

    View this table:
    Table 4

    Severity of adverse events

    Discussion

    The complexity of uncovering harm is reflected in the numerous ways that one has to measure it.17 Traditional methods such as incident reporting have limitations, especially that of under-reporting, due to the reliance on individual clinicians to recognise and report AEs, as well as a tendency to focus on error rather than on harm. The measurement and examination of harm, rather than that of error, recognises that efforts to improve patient safety benefit from focusing on incidents that result in actual harm, identifying high-risk situations, considering preventability and looking for means of early detection and harm limitation.18 This deeper understanding of the harm to which patients are exposed is a recent phenomenon and in paediatrics the potential risks to safety are multiple.19 The call for zero harm and the focus on safety in recent reports20 ,21 reflect the importance of the identification and understanding of harm as an essential part of patient care. Clinicians have not known the actual levels of harm caused and have relied on a reporting system for clinical incidents.

    Harm rates vary widely because of multiple factors, such as the definition of harm used, the methodology employed and the population studied. Until recently, most studies sought to establish harm rates for benchmarking purposes over large populations with suggestions that between 3% and 17% of patients experience an AE during a hospital admission.22 Most of these studies used retrospective, unstructured case note reviews which are labour-intensive, costly and impractical for the routine monitoring of harm.23 Trigger tool methodology is accepted as one of the ways to measure harm in a way that allows local learning,1 ,3 ,2 Trigger tools have also been reported to provide consistent, reliable and relevant data at low cost,2 ,3 although the cost may vary between different hospitals.

    This paper represents the largest study of paediatric harm using trigger tool methodology in the UK. While the primary purpose of a trigger tool is to gain local data for understanding harm in order to improve patient safety locally, we can learn about harm by examining the pooled experiences of the contributing hospitals.

    The overall harm rate (the percentage of individual children experiencing one or more AEs during an admission) identified in this study was 14.2%. Previous studies focusing on hospitalised children have identified harm rates between 1% and 25.8% per admission for the general paediatric hospital population,12 ,13 ,18 and higher rates within the paediatric intensive care population of 26.1% to 62%.10 ,11 ,24 Two recent studies have examined harm in paediatric hospital populations using trigger tool methodology. The CPTT found a physician who reported a harm rate of 15.1% of admissions during a validation study across six paediatric hospitals in Canada.25 Like the UKPTT, the CPTT has been adapted to make triggers more sensitive and specific to paediatric settings. The second study, at a single paediatric academic medical centre in the USA using the adult GTT, found an overall harm rate of 25.8%.8

    Variation in harm rates may reflect a number of factors. Different methodologies yield different rates of AE identification. Trigger tools are reported to yield higher rates of AE identification than traditional methods such as self-reporting and unstructured case note review.1 ,24 Definitions of harm vary, as do their interpretation. Professional groups may interpret AEs differently.26 Assessments of inter-rater reliability have reported high levels of agreement between review team members,8 ,26 ,27 but there is variability between different hospital department teams.28 In addition, some organisations or teams set a lower threshold for what they see as harm and they may change this over time. Finally, different populations are exposed to different levels of harm depending on the complexity of their illness and the intensity and duration of their care.2 Most studies report a harm rate per admission, meaning that longer admissions are more exposed to opportunity for harm.

    The same reasons that explain the variation between international studies also explain much of the variation between hospitals in this study. Training was provided, but no independent assessment was made of the reviewer's interpretations or competence. The extremes of harm reported or its absence were seen in hospitals uploading low volumes of reviews and may be interpreted as the relative inexperience of the reviewers.29 There is also a wide variety across the level of hospital represented with the corresponding impact on risk due to patient complexity, need for surgery or critical care and length of stay. While we had no means of adjusting for acuity because of the random selection of notes from within hospitals, we believe that the overall group is broadly representative of the population of hospitalised children in the UK.

    One could ask whether this level of variation diminishes the findings of the study. On the contrary, we believe it represents a real portrayal of complex issues. It is also a taste of what individual organisations can expect if they start to use the PTT to help understand and reduce the harm in their institution. They will need to consider all of these issues as they interpret their own findings.

    The majority (92.3%) of AEs identified in this study represented temporary harm resulting in the child requiring an intervention, admission to hospital or prolongation of their hospital stay. While severe harm (permanent harm or harm that required life-sustaining measures or contributed to death) was rare, it still constituted 7.8% of the harm identified. Similar findings with respect to severity have been reported with 10% of AEs classified as severe in one study of harm in a paediatric intensive care unit.10 A study of AEs in hospitalised children reported that clinicians do not always recognise harm, even when the consequences to the child are severe.30 In this study, multiple AEs were relatively common, with 37% of those experiencing harm suffering two or more AEs in the same admission, far higher than in previous studies.8 ,25

    Triggers varied in their PPV for AEs. Screening for triggers is the key task of the trigger tool method. Triggers that infrequently identify harm could be removed to increase the efficiency of the tool. Some triggers may be important markers of care quality, such as the missing/incomplete early warning score or baseline observations, despite the inability of the trigger to identify specific patient harm.31 This will influence the next iteration of the trigger tool as we refine the triggers and consider taking out some of those that had a low PPV.

    A number of studies have examined the possibility of automated trigger detection from electronic medical records, which may make the process easier.13 ,32 We believe that there is value in the manual approach, and that it will be some time before paper-based medical records in hospitals in the UK are converted to electronic medical records. Users of the UKPTT have expressed to us the benefits of having an opportunity to examine the quality of medical and nursing note keeping and observations, which in some centres has resulted in initiatives to improve these elements.

    Limitations

    There are a number of limitations to this study. The validity of trigger tool methodology is well established and we did not attempt to revalidate it again against another form of medical notes review for harm, as we did not believe this was necessary. The UKPTT differs from other trigger tools only in the constituent triggers. This study has provided PPVs for the 40 triggers included in the UKPTT. This validates the choice of high performing triggers and raises questions about the continued inclusion of low performing ones, which may be used to consider changes to the trigger profile. New triggers may also be suggested and could be tested for future versions of the tool.

    The determination of inter-rater reliability may be important within departments but not necessarily between hospitals. The UKPTT is not recommended for benchmarking as the focus is on developing data for improvement rather than data for judgement.33 The methodology recommends consistency in the reviewing teams so that intrareliability is not an issue.34 We did not attempt to standardise the method of PTT data collection outside of the support provided and the recommendation on randomisation. Individual institutions made their own arrangements in terms of choosing and training reviewers. There were no checks of competence of reviewers or inter-rater reliability or of the accuracy of the data entered via the portal.

    Strengths

    Parry et al35 note that the approach should be to look at all harm, not only preventable harm. It is our belief that the ability to measure harm and examine case notes using the UKPTT on a regular basis is an effective method of data capture and analysis, which provides hospitals with valuable insights into their quality of care, as every AE provides insight for improvement, whether deemed preventable or not.10

    We did not attempt to find out how many of these AEs were detected through other methods, such as incident reporting (we expect that many were). The purpose of the UKPTT is to extend the ability to detect harm rather than to replace other approaches.

    Conclusions and recommendations

    In the context of the increasing demands to improve quality and safety for patients, the UKPTT provides a framework for paediatric clinicians to assess the rate of harm for their individual units, and the quality of their record keeping. This study highlights that currently there is a significant, measurable level of harm, which is sometimes severe, experienced by children admitted to hospitals in the UK. There is a range of predictive values for the triggers and some may be more useful than others. These findings will inform future modifications of the PTT including modifying or removing triggers. It will be important to test any new or augmented triggers with paediatric teams to assess their usefulness.

    The recognition and examination of AEs through methods such as the UKPTT offers the potential to improve paediatric patient safety by concentrating efforts on strategies that reduce patient harm, rather than errors. The key is to produce information that promotes learning and improvement, with clinicians accepting their role to decrease harm from the perspective of the patient, rather than that of the healthcare provider.

    We recommend that the UKPTT be used routinely in hospitals to assess harm and to help develop improvement interventions to reduce it. Although the PTT has been mainly used in children's hospitals, it can be used in district general or community hospitals, with a different spectrum of harm being detected. The UKPTT does not replace other reporting mechanisms, but is a useful addition to the methods already used to understand the harm caused to children in hospital care. Harm needs to be detected and assessed through a number of lenses and this lens allows clinicians to further understand what they do and how harm impacts on children. It provides a way to move from a reactive approach to safety to one that is more proactive and founded on harm free care.

    Acknowledgments

    The authors would like to thank Matt Tite at the National Health Service (NHS) Institute for Innovation and Improvement for extracting and processing the data. The authors would also like to thank all the participating institutions whose unidentifiable data have been made available for the analysis, and those who participated in the development of the UKPTT.

    References

    1. ↵
      1. Resar RK,
      2. Rozich JD,
      3. Classen D
      . Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care 2003;12(Suppl 2):ii39–45.
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. Beyea SC
      . Using trigger tools to enhance patient safety. AORN J 2005;82:115–16.
      OpenUrlPubMed
    3. ↵
      1. Griffin FA,
      2. Resar RK
      . IHI Global Trigger Tool for measuring adverse events. 2nd edn. IHI Innovation Series white paper. Cambridge, MA: Institute for Healthcare Improvement, 2009.
    4. ↵
      1. Classen DC,
      2. Resar RK
      . The burden of harm. Jt Comm J Qual Patient Saf 2013:39:291.
      OpenUrl
    5. ↵
      1. Griffin FA,
      2. Classen DC
      . Detection of adverse events in surgical patients using the Trigger Tool approach. Qual Saf Health Care 2008;17:253–8.
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Resar RK,
      2. Rozich JD,
      3. Simmonds T,
      4. et al
      . A trigger tool to identify adverse events in the intensive care unit. Jt Comm J Qual Patient Saf 2006;32:585–90.
      OpenUrlPubMed
    7. ↵
      1. Singh RN,
      2. McLean-Plunckett EA,
      3. Kee R,
      4. et al
      . Experience with a trigger tool for identifying adverse drug events among older adults in ambulatory primary care. Qual Saf Health Care 2009;18:199–204.
      OpenUrlAbstract/FREE Full Text
    8. ↵
      1. Kirkendall ES,
      2. Kloppenborg E,
      3. Papp J,
      4. et al
      . Measuring adverse events and levels of harm in pediatric inpatients with the Global Trigger Tool. Pediatrics 2012;130:e1206–14.
      OpenUrlAbstract/FREE Full Text
    9. ↵
      1. Sharek PJ,
      2. Horbar JD,
      3. Mason W,
      4. et al
      . Adverse events in the neonatal intensive care unit: development, testing, and findings of an NICU-focused trigger tool to identify harm in North American NICUs. Pediatrics 2006;118:1332–40.
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Larsen GY,
      2. Donaldson AE,
      3. Parker HB,
      4. et al
      . Preventable harm occurring to critically ill children. Pediatr Crit Care Med 2007;8:331–6.
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Agarwal S,
      2. Classen D,
      3. Larsen G,
      4. et al
      . Prevalence of adverse events in pediatric intensive care units in the United States. Pediatr Crit Care Med 2010;11:568–78.
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      1. Takata GS,
      2. Mason W,
      3. Taketomo C,
      4. et al
      . Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children's hospitals. Pediatrics 2008;121:e927–35.
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Muething SE,
      2. Conway PH,
      3. Kloppenborg E,
      4. et al
      . Identifying causes of adverse events detected by an automated trigger tool through in-depth analysis. BMJ Qual Saf 2010;19:435–9.
      OpenUrlAbstract/FREE Full Text
    14. ↵
      1. Matlow AG,
      2. Cronin CMG,
      3. Flintoft V,
      4. et al
      . Description of the development and validation of the Canadian Paediatric Trigger Tool. BMJ Qual Saf 2011;20:416–23.
      OpenUrlAbstract/FREE Full Text
    15. ↵
    16. ↵
      1. Carter M
      . Measuring harm levels with the Global Trigger Tool. Clinical Risk 2010;16:122–6.
      OpenUrlAbstract/FREE Full Text
    17. ↵
      1. Thomas EJ,
      2. Petersen LA
      . Measuring harm and adverse events in healthcare. J Gen Intern Med 2003;18:61–7.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Woods D,
      2. Thomas E,
      3. Holl J
      , et al. Adverse events and preventable adverse events in children. Pediatrics 2005;115:155–60.
      OpenUrlAbstract/FREE Full Text
    19. ↵
      Review of patient safety for children and young people. National Patient Safety Agency 2009. http://www.nrls.npsa.nhs.uk/resources/?entryid45=59864 (accessed 25 May 2014).
    20. ↵
      Mid Staffordshire NHS Foundation Trust Public Inquiry. Report of the Mid Staffordshire NHS Foundation Trust Public Enquiry Feb 2013. http://www.midstaffspublicinquiry.com/report. (accessed 25 May 2014).
    21. ↵
      National Advisory Group on the Safety of Patients in England. A Promise to Learn a Committment to Act-Improving the safety of patients in England. 2013. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/226703/Berwick_Report.pdf (accessed 25 May 2014).
    22. ↵
      1. Sari ABA,
      2. Sheldon TA,
      3. Cracknell A,
      4. et al
      . Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. BMJ 2007:334:79.
      OpenUrlAbstract/FREE Full Text
    23. ↵
      1. Tinoco A,
      2. Evans RS,
      3. Staes CJ,
      4. et al
      . Comparison of computerized surveillance and manual chart review for adverse events. J Am Med Inform Assoc 2011;18:491–7.
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      1. Silas R,
      2. Tibballs J
      . Adverse events and comparison of systematic and voluntary reporting from a paediatric intensive care unit. Qual Saf Health Care 2010;19:568–71.
      OpenUrlAbstract/FREE Full Text
    25. ↵
      1. Matlow AG,
      2. Baker GR,
      3. Flintoft V,
      4. et al
      . Adverse Events among children in Canadian hospitals: the Canadian Paediatric Adverse Events Study. CMAJ 2012;184:E709–18).
      OpenUrlAbstract/FREE Full Text
    26. ↵
      1. Naessens JM,
      2. O'Byrne TJ,
      3. Johnson MG,
      4. et al
      . Measuring hospital adverse events: assessing inter-rater reliability and trigger performance of the Global Trigger Tool. Int J Qual Health Care 2010;22:266–74.
      OpenUrlAbstract/FREE Full Text
    27. ↵
      1. Sharek PJ,
      2. Parry GJ,
      3. Goldmann D,
      4. et al
      . Performance characteristics of a methodology to quantify adverse events over time in hospitalized patients. Health Serv Res 2011;46:654–78.
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      1. Zegers M,
      2. de Bruijne MC,
      3. Spreeuwenberg P,
      4. et al
      . Variation in the rates of adverse events between hospitals and hospital departments. Int J Qual Health Care 2011;23:126–33.
      OpenUrlAbstract/FREE Full Text
    29. ↵
      1. Zegers M,
      2. de Bruijne MC,
      3. Wagner C,
      4. et al
      . The inter-rater agreement of retrospective assessments of adverse events does not improve with two reviewers per patient record. J Clin Epidemiol 2010;63:94–102.
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      1. Woods DM,
      2. Holl JL,
      3. Shonkoff JP,
      4. et al
      . Child-specific risk factors and patient safety. J Patient Saf 2005;1:17–22.
      OpenUrlCrossRef
    31. ↵
      NHS Outcomes Framework 2012/13, Technical details of indicators. 2011: Section5.6:92. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/213713/dh_131721.pdf (accessed 25 May 2014).
    32. ↵
      1. Lemon V,
      2. Stockwell DC
      . Automated detection of adverse events in children. Pediatr Clin North Am 2012;59:1269–78.
      OpenUrlPubMed
    33. ↵
      1. Solberg LI,
      2. Mosser G,
      3. McDonald S
      . Three faces of performance measurement. J Qual Improv 1997:23:135–47.
      OpenUrl
    34. ↵
      1. Landrigan CP,
      2. Parry GJ,
      3. Bones CB,
      4. et al
      . Temporal trends in rates of patient harm resulting from medical care. N Engl J Med 2010;363:2124–34.
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      1. Parry GJ,
      2. Cline A,
      3. Goldmann DA
      . Deciphering harm measurement. JAMA 2012;307:2155–6.
      OpenUrlCrossRefPubMedWeb of Science

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • SMC and JF are joint first authors.

    • Contributors SMC were involved in the development of the Paediatric Trigger Tool, study concept, data collection, data analysis and manuscript preparation. JF involved in the development of paediatric trigger tool, study concept, data analysis and manuscript preparation. ND involved in the development of paediatric trigger tool, study concept, data collection, data analysis and manuscript revision. PL involved in the development of paediatric trigger tool, study concept, data analysis and manuscript revision.

    • Funding National Health Service (NHS) Institute for Innovation and Improvement.

    • Competing interests None.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.