Inter- and intra-rater reliability for classification of medication related events in paediatric inpatients
- 1School of Pharmacy, University of Otago, Dunedin, New Zealand
- 2Department of Women’s and Children’s Health, Dunedin School of Medicine, University of Otago, Dunedin, New Zealand
- 3Christchurch Women’s Hospital, Christchurch, New Zealand
- Correspondence to: MrsD L Kunac Research Fellow, School of Pharmacy, University of Otago, P O Box 913, Dunedin, New Zealand;
- Accepted 12 February 2006
Background: In medication safety research studies medication related events are often classified by type, seriousness, and degree of preventability, but there is currently no universally reliable “gold standard” approach. The reliability (reproducibility) of this process is important as the targeting of prevention strategies is often based on specific categories of event. The aim of this study was to determine the reliability of reviewer judgements regarding classification of paediatric inpatient medication related events.
Methods: Three health professionals independently reviewed suspected medication related events and classified them by type (adverse drug event (ADE), potential ADE, medication error, rule violation, or other event). ADEs and potential ADEs were then rated according to seriousness of patient injury using a seven point scale and preventability using a decision algorithm and a six point scale. Inter- and intra-rater reliabilities were calculated using the kappa (κ) statistic.
Results: Agreement between all three reviewers regarding event type ranged from “slight” for potential ADEs (κ = 0.20, 95% CI 0.00 to 0.40) to “substantial” agreement for the presence of an ADE (κ = 0.73, 95% CI 0.69 to 0.77). Agreement ranged from “slight” (κ = 0.06, 95% CI 0.02 to 0.10) to “fair” (κ = 0.34, 95% CI 0.30 to 0.38) for seriousness classifications but, by collapsing the seven categories into serious versus not serious, “moderate” agreement was found (κ = 0.50, 95% CI 0.46 to 0.54). For preventability decision, overall agreement was “fair” (κ = 0.37, 95% CI 0.33 to 0.41) but “moderate” for not preventable events (κ = 0.47, 95% CI 0.43 to 0.51).
Conclusion: Trained reviewers can reliably assess paediatric inpatient medication related events for the presence of an ADE and for its seriousness. Assessments of preventability appeared to be a more difficult judgement in children and approaches that improve reliability would be useful.
This research was supported by a Fellowship awarded to Desireé Kunac by the Child Health Research Foundation of New Zealand.
Competing interests: none.
Ethical approval for this study was granted by the Otago Ethics Committee.